DISCRETE TIME SYSTEMS 



Edited by Mario A. Jordan 
and Jorge L. Bustamante 



INTECHWEB.ORG 



Discrete Time Systems 

Edited by Mario A. Jordan and Jorge L. Bustamante 



Published by InTech 

Janeza Trdine 9, 51000 Rijeka, Croatia 

Copyright © 2011 InTech 

All chapters are Open Access articles distributed under the Creative Commons 
Non Commercial Share Alike Attribution 3.0 license, which permits to copy, 
distribute, transmit, and adapt the work in any medium, so long as the original 
work is properly cited. After this work has been published by InTech, authors 
have the right to republish it, in whole or part, in any publication of which they 
are the author, and to make other personal use of the work. Any republication, 
referencing or personal use of the work must explicitly identify the original source. 

Statements and opinions expressed in the chapters are these of the individual contributors 
and not necessarily those of the editors or publisher. No responsibility is accepted 
for the accuracy of information contained in the published articles. The publisher 
assumes no responsibility for any damage or injury to persons or property arising out 
of the use of any materials, instructions, methods or ideas contained in the book. 

Publishing Process Manager Ivana Lorkovic 

Technical Editor Teodora Smiljanic 

Cover Designer Martina Sirotic 

Image Copyright Emelyano, 2010. Used under license from Shutterstock.com 

First published March, 2011 
Printed in India 

A free online edition of this book is available at www.intechopen.com 
Additional hard copies can be obtained from orders@intechweb.org 



Discrete Time Systems, Edited by Mario A. Jordan and Jorge L. Bustamante 

p. cm. 
ISBN 978-953-307-200-5 



IKfTCn I OPENACCESS 
in I Lvn PUBLISHER 

INTECHopen 

free online editions of InTech 
Books and Journals can be found at 
www.intechopen.com 



Contents 



Preface IX 

Part 1 Discrete-Time Filtering 1 

Chapter 1 Real-time Recursive State Estimation 

for Nonlinear Discrete Dynamic Systems 
with Gaussian or non-Gaussian Noise 3 

Kerim Demirba§ 

Chapter 2 Observers Design for a Class of Lipschitz 

Discrete-Time Systems with Time-Delay 19 

Ali Zemouche and Mohamed Boutayeb 

Chapter 3 Distributed Fusion Prediction for Mixed 
Continuous-Discrete Linear Systems 39 

Ha-ryong Song, Moon-gu Jeon and Vladimir Shin 

Chapter 4 New Smoothers for Discrete-time Linear 

Stochastic Systems with Unknown Disturbances 53 

Akio Tanikawa 

Chapter 5 On the Error Covariance Distribution 

for Kalman Filters with Packet Dropouts 71 

Eduardo Rohr Damian Marelli, and Minyue Fu 

Chapter 6 Kalman Filtering for Discrete Time Uncertain Systems 93 

Rodrigo Souto, Joao Ishihara and Geovany Borges 

Part 2 Discrete-Time Fixed Control 109 

Chapter 7 Stochastic Optimal Tracking with Preview 

for Linear Discrete Time Markovian Jump Systems 111 

Gou Nakura 

Chapter 8 The Design of a Discrete Time Model Following 

Control System for Nonlinear Descriptor System 131 

Shigenori Okubo and Shujing Wu 



VI Contents 



Chapter 9 Output Feedback Control of Discrete-time 
LTI Systems: Scaling LMI Approaches 141 

Jun Xu 

Chapter 10 Discrete Time Mixed LQR/H„ Control Problems 159 

Xiaojie Xu 

Chapter 11 Robust Control Design of Uncertain 

Discrete-Time Systems with Delays 179 

Jun Yoneyama, Yuzu Uchida and Shusaku Nishikawa 

Chapter 12 Quadratic D Stabilizable Satisfactory Fault-tolerant 
Control with Constraints of Consistent Indices 
for Satellite Attitude Control Systems 195 

Han Xiaodong and Zhang Dengfeng 

Part 3 Discrete-Time Adaptive Control 205 

Chapter 13 Discrete-Time Adaptive Predictive Control 
with Asymptotic Output Tracking 207 

Chenguang Yang and Hongbin Ma 

Chapter 14 Decentralized Adaptive Control 

of Discrete-Time Multi-Agent Systems 229 

Hongbin Ma, Chenguang Yang and Mengyin Fu 

Chapter 15 A General Approach to Discrete-Time 

Adaptive Control Systems with Perturbed 
Measures for Complex Dynamics - Case Study: 
Unmanned Underwater Vehicles 255 

Mario Alberto Jordan and Jorge Luis Bustamante 

Part 4 Stability Problems 281 

Chapter 16 Stability Criterion and Stabilization 
of Linear Discrete-time System 
with Multiple Time Varying Delay 283 

Xie Wei 

Chapter 17 Uncertain Discrete-Time Systems with Delayed 
State: Robust Stabilization with Performance 
Specification via LMI Formulations 295 

Valter J. S. Leite, Michelle F. F. Castro, Andre F. Caldeira, 
Marcio F. Miranda and Eduardo N. Gongalves 

Chapter 18 Stability Analysis of Grey Discrete Time 

Time-Delay Systems: A Sufficient Condition 327 

Wen-Jye Shyr and Chao-Hsing Hsu 



Contents VII 



Chapter 19 Stability and £ 2 Gain Analysis of Switched Linear 
Discrete-Time Descriptor Systems 337 

Guisheng Zhai 

Chapter 20 Robust Stabilization for a Class of Uncertain 
Discrete-time Switched Linear Systems 351 

Songlin Chen, Yu Yao and Xiaoguan Di 

Part 5 Miscellaneous Applications 361 

Chapter 21 Half-overlap Subchannel Filtered MultiTone 
Modulation and Its Implementation 363 

Pavel Silhavy and Ondrej Krajsa 

Chapter 22 Adaptive Step-size Order Statistic LMS-based 
Time-domain Equalisation in Discrete 
Multitone Systems 383 

Suchada Sitjongsataporn and Peerapol Yuvapoositanon 

Chapter 23 Discrete-Time Dynamic Image-Segmentation System 405 

Ken'ichi Fujimoto, Mio Kobayashi and Tetsuya Yoshinaga 

Chapter 24 Fuzzy Logic Based Interactive Multiple Model 
Fault Diagnosis for PEM Fuel Cell Systems 425 

Yan Zhou, Dongli Wang, Jianxun Li, Lingzhi Yi and Huixian Huang 

Chapter 25 Discrete Time Systems with Event-Based Dynamics: 
Recent Developments in Analysis 
and Synthesis Methods 447 

Edgar Delgado-Eckert, Johann Reger and Klaus Schmidt 

Chapter 26 Discrete Deterministic and Stochastic Dynamical 
Systems with Delay - Applications 477 

Mihaela Neamtu and Dumitru Opris 

Chapter 27 Multidimensional Dynamics: 

From Simple to Complicated 505 

Kang-Ling Liao, Chih-Wen Shih and Jui-Pin Tseng 



Preface 



Discrete-Time Systems comprehend an important and broad research field. The con- 
solidation of digital-based computational means in the present, pushes a technological 
tool into the field with a tremendous impact in areas like Control, Signal Processing, 
Communications, System Modelling and related Applications. This fact has enabled 
numerous contributions and developments which are either genuinely original as 
discrete-time systems or are mirrors from their counterparts of previously existing 
continuous-time systems. 

This book attempts to give a scope of the present state-of-the-art in the area of Discrete- 
Time Systems from selected international research groups which were specially con- 
voked to give expressions to their expertise in the field. 

The works are presented in a uniform framework and with a formal mathematical 
context. 

In order to facilitate the scope and global comprehension of the book, the chapters were 
grouped conveniently in sections according to their affinity in 5 significant areas. 

The first group focuses the problem of Filtering that encloses above all designs of State 
Observers, Estimators, Predictors and Smoothers. It comprises Chapters 1 to 6. 

The second group is dedicated to the design of Fixed Control Systems (Chapters 7 to 
12). Herein it appears designs for Tracking Control, Fault-Tolerant Control, Robust Con- 
trol, and designs using LMI- and mixed LQR/Hoo techniques. 

The third group includes Adaptive Control Systems (Chapter 13 to 15) oriented to the 
specialities of Predictive, Decentralized and Perturbed Control Systems. 

The fourth group collects works that address Stability Problems (Chapter 16 to 20). 
They involve for instance Uncertain Systems with Multiple and Time-Varying Delays 
and Switched Linear Systems. 

Finally, the fifth group concerns miscellaneous applications (Chapter 21 to 27). They 
cover topics in Multitone Modulation and Equalisation, Image Processing, Fault Diag- 
nosis, Event-Based Dynamics and Analysis of Deterministic/Stochastic and Multidi- 
mensional Dynamics. 



X Preface 

We think that the contribution in the book, which does not have the intention to be 
all-embracing, enlarges the field of the Discrete-Time Systems with signification in the 
present state-of-the-art. Despite the vertiginous advance in the field, we think also that 
the topics described here allow us also to look through some main tendencies in the 
next years in the research area. 
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Real-time Recursive State Estimation for 

Nonlinear Discrete Dynamic Systems with 

Gaussian or non-Gaussian Noise 

Kerim Demirba§ 
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1. Introduction 

Many systems in the real world are more accurately described by nonlinear models. Since 
the original work of Kalman (Kalman, 1960; Kalman & Busy, 1961), which introduces the 
Kalman filter for linear models, extensive research has been going on state estimation 
of nonlinear models; but there do not yet exist any optimum estimation approaches for 
all nonlinear models, except for certain classes of nonlinear models; on the other hand, 
different suboptimum nonlinear estimation approaches have been proposed in the literature 
(Daum, 2005). These suboptimum approaches produce estimates by using some sorts of 
approximations for nonlinear models. The performances and implementation complexities 
of these suboptimum approaches surely depend upon the types of approximations which 
are used for nonlinear models. Model approximation errors are an important parameter 
which affects the performances of suboptimum estimation approaches. The performance of a 
nonlinear suboptimum estimation approach is better than the other estimation approaches for 
specific models considered, that is, the performance of a suboptimum estimation approach is 
model-dependent. 

The most commonly used recursive nonlinear estimation approaches are the extended 
Kalman filter (EKF) and particle filters. The EKF linearizes nonlinear models by Taylor 
series expansion (Sage & Melsa, 1971) and the unscented Kalman filter (UKF) approximates 
a posteriori densities by a set of weighted and deterministically chosen points (Julier, 2004). 
Particle filters approximates a posterior densities by a large set of weighted and randomly 
selected points (called particles) in the state space (Arulampalam et al., 2002; Doucet et al., 
2001; Ristic et al., 2004). In the nonlinear estimation approaches proposed in (Demirbas,, 
1982; 1984; Demirbas, & Leondes, 1985; 1986; Demirba§, 1988; 1989; 1990; 2007; 2010): the 
disturbance noise and initial state are first approximated by a discrete noise and a discrete 
initial state whose distribution functions the best approximate the distribution functions of the 
disturbance noise and initial state, states are quantized, and then multiple hypothesis testing 
is used for state estimation; whereas Grid-based approaches approximate a posteriori densities 
by discrete densities, which are determined by predefined gates (cells) in the predefined state 
space; if the state space is not finite in extent, then the state space necessitates some truncation 
of the state space; and grid-based estimation approaches assume the availability of the state 
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transition density p(x(k)\x(k — 1)), which may not easily be calculated for state models with 
nonlinear disturbance noise (Arulampalam et al., 2002; Ristic et al., 2004). The Demirbas, 
estimation approaches are more general than grid-based approaches since 1) the state space 
need not to be truncated, 2) the state transition density is not needed, 3) state models can be 
any nonlinear functions of the disturbance noise. 

This chapter presents an online recursive nonlinear state filtering and prediction scheme for 
nonlinear dynamic systems. This scheme is recently proposed in (Demirbas,, 2010) and is 
referred to as the DF throughout this chapter. The DF is very suitable for state estimation of 
nonlinear dynamic systems under either missing observations or constraints imposed on state 
estimates. There exist many nonlinear dynamic systems for which the DF outperforms the 
extended Kalman filter (EKF), sampling importance resampling (SIR) particle filter (which is 
sometimes called the bootstrap filter), and auxiliary sampling importance resampling (ASIR) 
particle filter. Section 2 states the estimation problem. Section 3 first discusses discrete noises 
which approximate the disturbance noise and initial state, and then presents approximate 
state and observation models. Section 4 discusses optimum state estimation of approximate 
dynamic models. Section 5 presents the DF. Section 6 yields simulation results of two 
examples for which the DF outperforms the EKF, SIR, and ASIR particle filters. Section 7 
concludes the chapter. 

2. Problem statement 

This section defines state estimation problem for nonlinear discrete dynamic systems. These 
dynamic systems are described by 

State Model 

x{k+l)=f(k,x(k),iu(k)) (1) 

Observation Model 

z(k) = g{k,x{k),v{k)), (2) 

where k stands for the discrete time index; / : RxR'"xR" — y R'" is the state transition function; 
R'" is the m-dimensional Euclidean space; w(k) £ R" is the disturbance noise vector at time 
k; x{k) g R m is the state vector at time k; g : RrR m xRP — > R'' is the observation function; 
v(k) £ RP is the observation noise vector at time k; z(k) £ W is the observation vector at time 
k; x(0), w(k), and v(k) are all assumed to be independent with known distribution functions. 
Moreover, it is assumed that there exist some constraints imposed on state estimates. The DF 
recursively yields a predicted value x (k\k — 1) of the state x(k) given the observation sequence 

from time one to time k — 1, that is, Z = {z(l), z(2), . . . ,z(/c — 1)}; and a filtered value 
x (k\k) of the state x(k) given the observation sequence from time one to time k, that is, Z . 
Estimation is accomplished by first approximating the disturbance noise and initial state with 
discrete random noises, quantizing the state, that is, representing the state model with a time 
varying state machine, and an online suboptimum implementation of multiple hypothesis 
testing. 



3. Approximation 

This section first discusses an approximate discrete random vector which approximates a 
random vector, and then presents approximate models of nonlinear dynamic systems. 
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3.1 Approximate discrete random noise 

In this subsection: an approximate discrete random vector with n possible values of a 
random vector is defined; approximate discrete random vectors are used to approximate 
the disturbance noise and initial state throughout the chapter; moreover, a set of equations 
which must be satisfied by an approximate discrete random variable with n possible values 
of an absolutely continuous random variable is given (Demirba§, 1982; 1984; 2010); finally the 
approximate discrete random variables of a Gaussian random variable are tabulated. 
Let w be an m-dimensional random vector. An approximate discrete random vector with n 
possible values of W, denoted by w d , is defined as an m-dimensional discrete random vector 
with n possible values whose distribution function the best approximates the distribution 
function of w over the distribution functions of all m-dimensional discrete random vectors 
with n possible values, that is 



Wa = mrn 



l {j^[Fy{a)-F w (a)] 2 da} (3) 



where D is the set of all m-dimensional discrete random vectors with n possible values, Fy(«) 
is the distribution function of the discrete random vector y, F w (a) is the distribution function 
of the random vector w, and R'" is the m-dimensional Euclidean space. An approximate 
discrete random vector w d is, in general, numerically, offline-calculated, stored and then used 
for estimation. The possible values of w d are denoted by w d \, Wgi, ••••, and w dn ; and the 
occurrence probability of the possible value w d j is denoted by P Wdi , that is 

P m = Prob{w d = w dl }. (4) 

where Prob{w d (0) = w d \) is the occurrence probability of Wfo. 

Let us now consider the case that w is an absolutely continuous random variable. Then, w d is 
an approximate discrete random variable with n possible values whose distribution function 
the best approximates the distribution function F w (a) of w over the distribution functions of 
all discrete random variables with n possible values, that is 

w d = mm- 1 {J(F y (a))} 
yeD 

in which the distribution error function (the objective function) J(Fy(a)) is defined by 



J(F y (a)) ± JjFy(a) - F w (a)} 



2 da 



where D is the set of all discrete random variables with n possible values, Fy(a) is the 
distribution function of the discrete random variable y, F w (a) is the distribution function of the 
absolutely continuous random variable w, and R is the real line. Let the distribution function 
Fy(a) of a discrete random variable y be given by 

if fl < i/l 

if y,- < a < yi + \, i = 1, 2, . . . , n — 1 

if a > y„ . 

Then the distribution error function J(Fy(a)) can be written as 

/J/l W — 1 /■1/i+l TOO 

Fl{a)da + X] / [F y , - F w (a)] 2 da + / [1 - F w (a)] 2 da. 
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Let the distribution function F Wi (a) of an approximate discrete random variable w d be 

A ( if a < w dl 

Fw d (a) = { F m if w di < a < w di+1 , i = 1, 2, . . . , n - 1 






if a < w d \ 


Fw di 


if w dl <a< w dl+l , i 


1 


iftf > w dn . 



It can readily be shown that the distribution function F Wil (a) of the approximate discrete 
random variable w d must satisfy the set of equations given by 

F Wn =2F w (w dl ); 
Fw d , + F Wim =2F w {w di+l ), i = 1, 2, . . . , n - 2; (5) 

1 + Fm„-i=2Fw(wdn); 
Fwm [v><a+l - w di\ = / Fw {a)da, i = \,2, .... 



1. 



The values w dl , w d2 , ■■■, w dn , F Wil , F Wil , ■■■,F Wjn satisfying the set of Eqs. (5) determine the 
distribution function of w d . These values can be, in general, obtained by numerically solving 
Eqs. (5). Then the possible values of the approximate discrete random variable io d become 
w d\r w d2r ■■■' an d w dn i an d the occurrence probabilities of these possible values are obtained 
by 

(F Wdl ifi = l 

Fw di = < F Wii - F Wii _, if i = 2, 3, ...,«- 1 

[l-F- a , dil if i = n. 

where P Wdi = Prob{w d = w d j}, which is the occurrence probability of w dl . 
Let y be a Gaussian random variable with zero mean and unit variance. An approximate 
discrete random variable y d with n possible values was numerically calculated for different 
n's by using the set of Eqs. (5). The possible values y d \, y d2 , ■■■, y dn of y d and the 
occurrence probabilities Py dl , Py dl , ■■■, Fy d „ of these possible values are given in Table 1, where 

Py di = Prob{y d = y d j}. As an example, the possible values of an approximate discrete 
random variable with 3 possible values of a Gaussian random variable with zero mean and 
unit variance are -1.005, 0.0, and 1.005; and the occurrence probabilities of these possible 
values are 0.315, 0.370, and 0.315, respectively. Let w be a Gaussian random variable with 
mean E{w} and variance cr 2 . This random variable can be expressed as if = ya + E{w}. 
Hence, the possible values of an approximate discrete random variable of w are given by 
w di = ydi a + E{io}, where i = 1, 2, 3, ..., n; and the occurrence probability of the possible value 
w d i is the same as the occurrence probability of y d j, which is given in Table 1. 

3.2 Approximate models 

For state estimation, the state and observation models of Eqs. (l)and (2) are approximated by 
the time varying finite state model and approximate observation model which are given by 

Finite State Model 

x q (k+l) = Q(f(k,x q (k),w d (k))) (6) 

Approximate Observation Model 

z(k) = g(k,x q (k),v(k)), (7) 
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n 


Veil Vdl Vd3 Vdi Vd5 Vdb x Jd7 VdS Vd9 3/dl0 

-fj/ii Py<u Pyd3 Py<n P'jd5 Pyd6 ^W? Pym Pyts Py<iw 


1 


0.000 
1.000 


2 


-0.675 0.675 
0.500 0.500 


3 


-1.005 0.0 1.005 
0.315 0.370 0.315 


4 


-1.218 -0.355 0.355 1.218 
0.223 0.277 0.277 0.223 


5 


-1.377-0.592 0.0 0.592 1.377 
0.169 0.216 0.230 0.216 0.169 


6 


-1.499 -0.768 -0.242 0.242 0.768 1.499 
0.134 0.175 0.191 0.191 0.175 0.134 


7 


-1.603 -0.908 -0.424 0.0 0.424 0.908 1.603 
0.110 0.145 0.162 0.166 0.162 0.145 0.110 


8 


-1.690 -1.023 -0.569 -0.184 0.184 0.569 1.023 1.690 
0.092 0.124 0.139 0.145 0.145 0.139 0.124 0.092 


9 


-1.764 -1.120 -0.690 -0.332 0.332 0.690 1.120 1.764 
0.079 0.106 0.121 0.129 0.130 0.129 0.121 0.106 0.079 


10 


-1.818 -1.199 -0.789 -0.453 -0.148 0.148 0.453 0.789 1.199 1.818 
0.069 0.093 0.106 0.114 0.118 0.118 0.114 0.106 0.093 0.069 



Table 1. Approximate Discrete Random Variables the best Approximating the Gaussian 
Random Variable with Zero Mean and Unit Variance 

where w,j(k) is an approximate discrete random vector with, say, n possible values of the 
disturbance noise vector w(k); this approximate vector is pre(offline)-calculated, stored and 
then used for estimation to calculate quantization levels at time k + 1; the possible values of 
iv c i(k) are denoted by w^k), iv d2 (k), ■■■■, and w l f n (k) ; Q : W — )■ R"' is a quantizer which 
first divides the m-dimensional Euclidean space into nonoverlapping generalized rectangles 
(called gates) such that the union of all rectangles is the m-dimensional Euclidean space, and 
then assigns to each rectangle the center point of the rectangle, Fig. 1 (Demirba§, 1982; 1984; 
2010); Xq(k), k > 0, is the quantized state vector at time k and its quantization levels, whose 
number is (say) mj-, are denoted by x„\(k), x,o{k), ...., and Xq mk (k). The quantization levels 
of Xq(k + 1) are calculated by substituting Xq (k) = x„j(k) (i = 1, 2, . . . , m^) for Xq (k) and 
w d{k) = Wdj(k) = 1/2,..., n) for Wd(k) in the finite state model of Eq. (6). As an example, 
let the quantization level x„i(k) in the gate G, be mapped into the gate G, by the / -possible 
value Wcn{k) of w d (k), then, x(k + 1) is quantized to XqAk + 1), Fig. 1. One should note that 
the approximate models of Eqs. (6) and (7) approach the models of Eqs. (1) and (2) as the gate 
sizes {GS) —¥ and n — > oo. An optimum state estimation of the models of Eqs. (6) and (7) is 
discussed in the next section. 



4. Optimum state estimation 

This section discuses an optimum estimation of the models of Eqs. (6) and (7) by using 
multiple hypothesis testing. On the average overall error probability sense, optimum 
estimation of states of the models of Eqs. (6) and (7) is done as follows: Finite state model 
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Fig. 1. Quantization of States 

of Eq. (6) is represented by a trellis diagram from time to time k (Demirba§, 1982; 1984; 
Demirbas, & Leondes, 1985; Demirbas,, 2007). The nodes at time j of this trellis diagram 
represent the quantization levels of the state x(j) . The branches of the trellis diagram represent 
the transitions between quantization levels. There exist, in general, many paths through this 
trellis diagram. Let H' denote the i path (sometimes called the i hypothesis) through the 
trellis diagram. Let xl(j) be the node (quantization level) through which the path H' passes 
at time j. The estimation problem is to select a path (sometimes called the estimator path) 
through the trellis diagram such that the average overall error probability is minimized for 
decision (selection). The node at time k along this estimator path will be the desired estimate 
of the state x(k). In Detection Theory (Van Trees, 2001; Weber, 1968): it is well-known that the 
optimum decision rule which minimizes the average overall error probability is given by 



Select H" as the estimator path if M(H n ) > M{H l ) for all I ^ n, 
where M(H n ) is called the metric of the path M(H n ) and is defined by 

M(H") = \n{p(H")Prob{observation sequence \ H n }}, 



(8) 



(9) 



where In stands for the natural logarithm, p(H n ) is the occurrence probability (or the a 
priori probability) of the path H", and Prob {observation sequence \ H' 1 } is the conditional 
probability of the observation sequence given that the actual values of the states are equal 
to the quantization levels along the path H" . If the inequality in the optimum decision rule 
becomes an equality for an observation sequence, anyone of the paths satisfying the equality 
can be chosen as the estimator path, which is a path having the biggest metric. 
It follows, from the assumption that samples of the observation noise are independent, that 
Prob {observation sequence \ H n } can be expressed as 



Prob{observation sequence \ H"} = TJ[A(z(;) | x"(j)) 



(10) 
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where 

. ... n\/-\\ A fl if z(j) is neither available nor used for estimation 

I {])\ 1'V'''\p(z(j)\Xq(j)) if z(j) is available and used for estimation, ^ ' 

in which, p(z(j)\x"(j)) is the conditional density function of z(j) given that the actual value 
of state is equal to x'l(j), that is, x(j) = x'l(j); and this density function is calculated by using 
the observation model of Eq. (2). 

It also follows, from the assumption that all samples of the disturbance noise and the initial 
state are independent, that the a priori probability of H n can be expressed as 

p(H n ) = P«b{x,(0) = *?(0)>nn*?0'- 1) -► *?(/)), (12) 

7=1 

where Prob{xg(0) = x"(0)} is the occurrence probability of the initial node (or quantization 
level) x"(0), and T(x"(j — 1) — y x"(j)) is the transition probability from the quantization 

level x'Uj — 1) to the quantization level x"(j)), that is, T(xUi — 1) — > x"(j)) = Prob{Xq(j) = 
x q(j)\ x q(j — 1) = x qU ~ 1)}/ which is the probability that x"(; — 1) is mapped to x"(j) by 
the finite state model of Eq. (6) with possible values of W&(j — 1). Since the transition from 
x q(] ~ 1) to x q{j) is determined by possible values of W^(j — 1), this transition probability is 
the sum of occurrence probabilities of all possible values of Wjij — 1) which map x n „ (j — 1) to 

The estimation problem is to find the estimator path, which is the path having the biggest 
metric through the trellis diagram. This is accomplished by the Viterbi Algorithm (Demirba§, 
1982; 1984; 1989; Forney, 1973); which systematically searches all paths through the trellis 
diagram. The number of quantization levels of the finite state model, in general, increases 
exponentially with time k. As a result, the implementation complexity of this approach 
increases exponentially with time k (Demirbas,, 1982; 1984; Demirbas, & Leondes, 1985; 
Demirba§, 2007). In order to overcome this obstacle, a block-by-block suboptimum estimation 
scheme was proposed in (Demirbas,, 1982; 1984; Demirba§ & Leondes, 1986; Demirba§, 1988; 
1989; 1990). In this estimation scheme: observation sequence was divided into blocks of 
constant length. Each block was initialized by the final state estimate from the last block. The 
initialization of each block with only a single quantization level (node), that is, the reduction 
of the trellis diagram to one node at the end of each block, results in state estimate divergence 
for long observation sequences, i.e., large time fc, even though the implementation complexity 
of the proposed scheme does not increase with time (Kee & Irwin, 1994). The online and 
recursive state estimation scheme which is recently proposed in (Demirba§, 2010) prevents 
state estimate divergence caused by one state initialization of each block for the block-by-block 
estimation. This recently proposed estimation scheme, referred to as the DF throughout 
this chapter, first prunes all paths going through the nodes which do not satisfy constraints 
imposed on estimates and then assigns a metric to each node (or quantization level) in the 
trellis diagram. Furthermore, at each time (step, or iteration), the number of considered state 
quantization levels (nodes) is limited by a selected positive integer MN, which stands for 
the maximum number of quantization levels considered through the trellis diagram; in other 
words , MN nodes having the biggest metrics are kept through the trellis diagram and all the 
paths going through the other nodes are pruned. Hence, the implementation complexity of 
the DF does not increase with time. The number MN is one of the parameters determining 
the implementation complexity and the performance of the DF. 
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5. Online state estimation 

This section first yields some definitions, and then presents the DF. 

5.1 Definitions 

Admissible initial state quantization level : a possible value x„j(0) = x^,(0) of an 

approximate discrete random vector Xq(0) = x d (0) of the initial state vector x(0) is said 
to be an admissible quantization level of the initial state vector (or an admissible initial 
state quantization level) if this possible value satisfies the constraints imposed on the state 
estimates. Obviously, if there do not exist any constraints imposed on the state estimates, 
then all possible values of the approximate discrete random vector x q (0) are admissible. 

Metric of an admissible initial state quantization level: the natural logarithm of the 
occurrence probability of an admissible initial quantization level x„,(0) is referred to as the 
metric of this admissible initial quantization level. This metric is denoted byM(i„i(0)), that 
is 

M(x,,,(0)) = \n{Prob{x q (0) = x ql (0)}}. (13) 

where Prob{xq(0) = x„,(0)} is the occurrence probability of x„j(0). 

Admissible state quantization level at time k: a quantization level x„{(k) of a state vector 
x(k), where fc > 1, is called an admissible quantization level of the state (or an admissible 
state quantization level) at time k if this quantization level satisfies the constraints imposed 
on the state estimates. Surely, if there do not exist any constraints imposed on the state 
estimates, then all the quantization levels of the state vector x(k), which are calculated by Eq. 
(6), are admissible. 

Maximum number of considered state quantization levels at each time: MN stands for the 
maximum number of admissible state quantization levels which are considered at each time 
(step or iteration) of the DF. MN is a preselected positive integer. A bigger value of MN yields 
better performance, but increases implementation complexity of the DF. 
Metric of an admissible quantization level (or node) at time k, where k > 1: the metric of an 
admissible quantization level x„,(fc), denoted by M(x„,(fc)), is defined by 

M(x qj (k))=max{M{ Xqtl (k - 1)) + ln[T(x q „(k - 1) -> *,/(*))]} 

+ ln[A(z(k)\x qj (k))}, (14) 

where the maximization is taken over all considered state quantization levels at time fc — 1 
which are mapped to the quantization level x„j(k) by the possible values of w d {k — 1); In 
stands for the natural logarithm; T(x qn (k — 1) — > x„j(k)) is the transition probability from 
x ql {k - 1) to x q j(k) is given by 

T(x ql (k-1) -> x qj (k)) = YJ>rob{w d {k-l) = w dn (k-l)}, (15) 

ii 

where Prob{w l i(k — 1) = w dn (k — 1)} is the occurrence probability of w dn {k — 1) and the 
summation is taken over all possible values of w d (k — 1) which maps x„j(k — 1) to x„j(k); in 



Real-time Recursive State Estimation for Nonlinear 

Discrete Dynamic Systems with Gaussian or non-Gaussian Noise 1 1 

other words, the summation is taken over all possible values of W^(k — 1) such that 

Q(f{k-l,x ql {k-l),w dn (k-l))) = x^k); (16) 

and 

, , ,,.. .... a [1 if z(j) is neither available nor used for estimation . 

I I )\ qj\ )' ~\-p(z{k)\x t ,j{k)) if z(j) is available and used for estimation, ^ 

in which, p(z(k)\x„;(k)) is the conditional density function of z(k) given that the actual value 
of state x(k) = x„j(k), and this density function is calculated by using the observation model 
of Eq. (2). 

5.2 Estimation scheme (DF) 

A flowchart of the DF is given in Fig. 3 for given F m m (a), Fwo) ( a )/ MN, n, m , and GS; where 
^w(k) ( a ) an d ^x(O) ( fl ) are * ne distribution functions of w(k) and x(0) respectively, n and m are 
the numbers of possible values of approximate random vectors of w (k) and x(0) respectively; 
GS is the gate size; and z(k) is the observation at time k. The parameters MN, n, m , and 
GS determine the implementation complexity and performance of the DF. The number of 
possible values of the approximate disturbance noise W&(k) is assumed to be the same, n , for 
all iterations, i.e., for all k. The filtered value x(k\k) and predicted value x(k\k — 1) of the state 
x(k) are recursively determined by considering only MN admissible state quantization levels 
with the biggest metrics and discarding other quantization levels at each recursive step (each 
iteration or time) of the DF. Recursive steps of the DF is described below. 

Initial Step (Step 0): an approximate discrete random vector -Trf(O) with m possible values of 
the initial state x(0) is offline calculated by Eq. (3). The possible values of this approximate 
random vector are defined as the initial state quantization levels (nodes). These initial state 

quantization levels are denoted by x„ 1 (0), ^(O), •••/ and Xq,„(0), where x„M) = *d;(o) (* = 
1 2 ...m). Admissible initial state quantization levels, which satisfy the constraints imposed 
on state estimates, are determined and the other initial quantization levels are discarded. If 
the number of admissible initial quantization levels is zero, then the number, m, of possible 
values of the approximate initial random vector x d (0) is increased and the initial step of the 
DF is repeated from the beginning; otherwise, the metrics of admissible initial quantization 
levels are calculated by Eq. (13). The admissible initial state quantization levels (represented 
by x„j(0), x„2(0), — / an d *oN (0)) and their metrics are considered in order to calculate state 
quantization levels and their metrics at time k = 1. These considered quantization levels are 
denoted by nodes (at time 0) on the first row (or column) of two rows (or columns) trellis 
diagram at the first step k = 1 of the DF, Fig. 2. 

State estimate at time 0: if the mean value of x(0) satisfies constraints imposed on state estimates 
such as the case that there do not exist any estimate constraints , then this mean value is taken 
as both x(0|0) and x(0|0 — 1); otherwise, the admissible initial state quantization level (node) 
with the biggest metric is taken as both the filtered value x(0|0) and predicted value x (0|0 — 1) 
of the state x(0), given no observation. 

Recursive Step (Step k): An approximate discrete disturbance noise vector w^(k — 1) with 
n possible values of the disturbance noise wik — 1) is offline obtained by Eq. (3). The 
quantization levels of the state vector at time k are calculated by using the finite state model 
of Eq. (6) with all the considered quantization levels (or nodes) Xn\{k — 1), x„2{k — 1) ... 
x qN k -i{k — 1) at time k — 1; and all possible values Wfn{k — 1), zv^ik — 1), ..., w dn (k — 1) 
of the approximate discrete disturbance noise vector w^(k — 1) . That is, substituting the 
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Fig. 2. Two Row Trellis Diagram of Admissible State Quantization Levels 

considered state quantization levels %„{{k— 1) (i = 1, 2, . . . ,N^_\) for x„(k — 1) and the 
possible values w d (k — 1) = Wrf,(fc — 1) (/' = 1, 2, . . . ,n) for H^(fc — 1) in the finite state 
model of Eq. (6), the quantization levels of the state at time k are calculated (generated). The 
admissible quantization levels at time k, which satisfy constraints imposed on state estimates, 
are determined and non-admissible state quantization levels are discarded. If the number 
of admissible state quantization levels at time k is zero, then a larger n, MN or smaller GS 
is taken and the recursive step at time k of the DF is repeated from the beginning; otherwise, 
the metrics of all admissible state quantization levels at time k are calculated by using Eq. 
(14). If the number of admissible state quantization levels at time k is greater than MN, then 
only MN admissible state quantization levels with biggest metrics, otherwise, all admissible 
state quantization levels with their metrics are considered for the next step of the DF. The 
considered admissible quantization levels (denoted by x„i{k), x„2(k), ...,x„-^ k (k)) and their 
metrics are used to calculate the state quantization levels and their metrics at time k + 1. The 
considered state quantization levels at time k are represented by the nodes on the second row 
(or column) of two rows (or columns) trellis diagram at the recursive step k and on the first row 
(or column) of two rows (or columns) trellis diagram at the recursive step k + 1, Fig. 2; where 
the subscript Njt, which is the number of considered nodes at the end of Recursive step k, is less 
than or equal to MN; and the transition from a node at time k — 1, say x„i(k — 1), to a node at 
time k , say x„j(k), is represented by a directed line which is called a branch. Estimate at time 
k: the admissible quantization level (node) with the biggest metric at time k is taken as the 
desired estimate of the state at time k, that is, the node with the biggest metric at time k is the 
desired predicted value of x(k) if z(k) is neither available nor used for estimation; otherwise, 
the node at time k with the biggest metric is the filtered value of x(k). If there exist more than 
one nodes having the same biggest metric, anyone of these nodes can be taken as the desired 
estimate. 
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Fig. 3. Flowchart of the DF 
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Fig. 4. Average Filtering Errors for Eqs. (18) and (19) 

6. Simulations 

In this section, Monte Carlo simulation results of two examples are given. More examples are 
presented in (Demirbas,, 2010). The first example is given by 



State Model 

x(k + 1) = x(jfc) [1 + t^— cos(0.8x(fc) + 2w{k))\ + w{k) 
K -j- 1 

Observation Model 



z(k) 



6x{k) 
l + x 2 {k) 



v(k), 



(18) 
(19) 



where the random variables x(0), w(k), and v(k) are independent Gaussian random variables 
with means 6, 0, and variances 13, 20, 15 respectively. It was assumed that there did not 
exist any constraints imposed on state estimates. The state model of Eq. (18) is an highly 
nonlinear function of the disturbance noise w(k). The extended Kalman filter (EKF) and the 
grid-based approaches may not be used for the state estimation of this example, since the 
EKF assumes a linear disturbance noise in the state model and the grid based approaches 
assumes the availability of the state transition density p(x(k)\x(k — 1)) which may not readily 
calculated (Arulampalam et al., 2002; Ristic et al., 2004). States of this example were estimated 
by using the DF, the sampling importance resampling (SIR) particle filter (which is sometimes 
called the bootstrap filter, and auxiliary sampling importance resampling (ASIR) particle filter 
(Arulampalam et al., 2002; Gordon et al., 1993). Average absolute filtering and prediction 
errors are sketched in Figs. 4 and 5 for 2000 runs each of which consists of 100 iterations. 
These estimation errors were obtained by using the SIR and ASIR particle filters with 1000 
particles and the DF for which the random variables x(0) and w(k) were approximated by the 
approximate random variables with 3 possible values (which are given in Section 3); the gate 
size (GS) and MN were taken as 0.1 and 8 respectively. The average filtering and prediction 
errors per one estimation (one iteration) were 33.8445, 45.6377, 71.5145 and 34.0660, 45.4395, 
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Fig. 5. Average Prediction Errors for Eqs. (18) and (19) 

70.2305 respectively. A typical run with 100 iteration took 0.0818, 0.2753, 0.3936 seconds for 
the DF, SIR and ASIR particle filters, respectively. The DF clearly performs better than both 
the SIR and ASIR particle filter. Moreover, the DF is much faster than both the SIR and ASIR 
particle filters with 1000 particles. 
The second example is described by 



State Model 

x(k + 1) = x(k)[l + -A- cos(0.8x(Jfc))] + w(Jfc) 

K -\- 1 
Observation Model 
6x(k) 



z(k) 



1 + **(*) 



■v(k), 



(20) 
(21) 



where the random variables x(0), w(k),and v(k) are independent Gaussian random variables 
with means 3, 0, and variances 8, 9, 9 respectively. It was assumed that there did not 
exist any constraints imposed on state estimates. Average absolute filtering and prediction 
errors are sketched in Figs. 6 and 7 for 2000 runs each of which consists of 200 iterations. 
These estimation errors were obtained by using the SIR and ASIR particle filters with 1000 
particles and the DF for which the random variables x(0) and w(k) were approximated by the 
approximate random variables with 3 possible values (which are given in Section 3); the gate 
size (GS) and MN were taken as 0.1 and 4 respectively. The average filtering and prediction 
errors per one estimation (one iteration) were 38.4913, 61.5432, 48.4791 and 38.5817, 61.4818, 
48.5088 respectively. A typical run with 200 iteration took 0.0939, 0.5562, 0.8317 seconds for 
the DF, SIR and ASIR particle filters, respectively. The state model of the second example is 
a linear function of the disturbance noise. Hence, the extended Kalman filter (EKF) was also 
used for state estimation, but the EKF estimation errors quickly diverged, hence, the EKF state 
estimation errors are not sketched. The DF clearly performs better than the EKF, SIR and ASIR 
particle filters and also the DF is much faster than both the SIR and ASIR particle filters with 
1000 particles for the second example. 
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Fig. 7. Average Prediction Errors for Eqs. (20) and (21) 

The performance of the DF is determined by the possible values (n and m) of the approximate 
discrete random disturbance noise and approximate discrete initial state, gate size (GS), 
maximum number (MN) of considered state quantization levels at each iteration. As GS goes 
to zero and the parameters n, m, and MN approach infinity, the approximate models of Eq. 
(6) and (7) approach the models of Eqs. (1) and (2), hence, the DF approaches an optimum 
estimation scheme, but the implementation complexity of the DF exponentially increases with 
time k. The parameters n, m, GS, MN which yield the best performance for given models 
are determined by Monte Carlo simulations for available hardware speed and storage. For 
given nonlinear models: the performances of the DF, EKF, particle filters, and others must 
be compared by Monte Carlo simulations with available hardware speed and storage. The 
estimation scheme yielding the best performance should be used. The EKF is surely much 
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faster than both the DF and particle filters. The speed of the DF is based upon the parameters 
n, m, GS, MN; whereas the speeds of particle filters depend upon the number of particles 
used. 

7. Conclutions 

Presented is a real-time (online) recusive state filtering and prediction scheme for nonlinear 
discrete dynamic systems with Gaussian or non-Gaussian disturbance and observation noises. 
This scheme, referred to as the DF, is recently proposed in (Demirba§, 2010). The DF is very 
suitable for state estimation of nonlinear dynamic systems under either missing observations 
or constraints imposed on state estimates. The DF is much more general than grid based 
estimation approaches. This is based upon discrete noise approximation, state quantization, 
and a suboptimum implementation of multiple hypothesis testing , whereas particle filters 
are based upon sequential Monte Carlo Methods. The models of the DF is as general as 
the models of particle filters, whereas the models of the extended Kalman filter (EKF) are 
linear functions of the disturbance and observation noises. The DF uses state models only to 
calculate transition probabilities from gates to gates. Hence, if these transition probabilities 
are known or can be estimated, state models are not needed for estimation with the DF, 
whereas state models are needed for both the EKF and particle filters. The performance 
and implementation complexity of the DF depend upon the gate size (GS), numbers n and 
m of possible values of approximate discrete disturbance noise and approximate discrete 
initial state, and maximum number (MN) of considered quantization levels at each iteration 
of the DF; whereas the performances and implementation complexities of particle filters 
depend upon numbers of particles used. The implementation complexity of the DF increases 
with a smaller value of GS, bigger values of n, m, and MN. These yield more accurate 
approximations of state and observation models; whereas the implementation complexities 
of particle filters increase with larger numbers of particles, which yield better approximations 
of conditional densities. Surely, the EKF is the simplest one to implement. The parameters 
(GS, n,m, MN) for which the DF yields the best performance for a real-time problem should 
be determined by Monte Carlo simulations. As GS — ► 0, n — ¥ oo, m — ¥ oo,and MN —¥ oo; 
the DF approaches the optimum one in the average overall error probability sense, but its 
implementation complexity exponentially increases with time. The performances of the DF, 
particle filters, EKF are all model-dependent. Hence, for a real-time problem with available 
hardware speed and storage; the DF, particle filters, and EKF (if applicable) should all be 
tested by Monte Carlo simulations, and the one which yields the best results should be used. 
The implementation complexity of the DF increases with the dimensions of multidimensional 
systems, as in the particle filters. 
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1. Introduction 

The observer design problem for nonlinear time-delay systems becomes more and 
more a subject of research in constant evolution Germani et al. (2002), Germani & 
Pepe (2004), Aggoune et al. (1999), Raff & Allgower (2006), Trinh et al. (2004), Xu et al. 
(2004), Zemouche et al. (2006), Zemouche et al. (2007). Indeed, time-delay is frequently 
encountered in various practical systems, such as chemical engineering systems, neural 
networks and population dynamic model. One of the recent application of time-delay is 
the synchronization and information recovery in chaotic communication systems Cherrier 
et al. (2005). In fact, the time-delay is added in a suitable way to the chaotic system in the 
goal to increase the complexity of the chaotic behavior and then to enhance the security of 
communication systems. On the other hand, contrary to nonlinear continuous-time systems, 
little attention has been paid toward discrete-time nonlinear systems with time-delay. We 
refer the readers to the few existing references Lu & Ho (2004a) and Lu & Ho (2004b), where 
the authors investigated the problem of robust Hoo observer design for a class of Lipschitz 
time-delay systems with uncertain parameters in the discrete-time case. Their method show 
the stability of the state of the system and the estimation error simultaneously. 
This chapter deals with observer design for a class of Lipschitz nonlinear discrete-time 
systems with time-delay. The main result lies in the use of a new structure of the proposed 
observer inspired from Fan & Arcak (2003). Using a Lyapunov-Krasovskii functional, a 
new nonrestrictive synthesis condition is obtained. This condition, expressed in term of 
LMI, contains more degree of freedom than those proposed by the approaches available in 
literature. Indeed, these last use a simple Luenberger observer which can be derived from the 
general form of the observer proposed in this paper by neglecting some observer gains. 
An extension of the presented result to H^ performance analysis is given in the goal to 
take into account the noise which affects the considered system. A more general LMI is 
established. The last section is devoted to systems with differentiable nonlinearities. In 
this case, based on the use of the Differential Mean Value Theorem (DMVT), less restrictive 
synthesis conditions are proposed. 

Notations : The following notations will be used throughout this chapter. 
• II . || is the usual Euclidean norm; 
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• (*) is used for the blocks induced by symmetry; 

• A represents the transposed matrix of A; 

• I r represents the identity matrix of dimension r; 

• for a square matrix S, S > (S < 0) means that this matrix is positive definite (negative 
definite); 

• Zf (k) represents the vector x(k — t) for all z; 



• The notation \\x\\p = ( E£Lo ll*MII J i s me ^f norrn OI m e vector x 6 R s . The set l s 2 is 
defined by 

4 = Ix GR S : ||x||^ < +oo}. 

2. Problem formulation and observer synthesis 

In this section, we introduce the class of nonlinear systems to be studied, the proposed state 
observer and the observer synthesis conditions. 

2.1 Problem formulation 

Consider the class of systems described in a detailed forme by the following equations : 

x(k + l) = Ax(k) + A d x d (k) + Bf(Hx(k),H d x d (k)) (la) 

y(k) = Cx(k) (lb) 

x(k) = x°(k), for k = -d,...,0 (lc) 

where the constant matrices A, A d , B, C, H and H d are of appropriate dimensions. 

The function / : R Sl x R S2 — > R 1 ? satisfies the Lipschitz condition with Lipschitz constant jr, 

i.e : 

zi-zi 

_z 2 - z 2 _ 

Now, consider the following new structure of the proposed observer defined by the 
equations (78) : 



||/(zi,z 2 ) — /(zi,Z2) || < 7/ 



, Vz 1 ,z 2 ,z 1 ,z 2 . (2) 



x(k + 1) = Ax(k) + A d x d (k) + Bf(v(k),w(k)] 

+ L(y(k)-Cx(k))+L d (y d (k)-Cx d (k)) 



(3a) 



v{k) = Hx(k) + K 1 (y{k) - Cx(k)) + K] (y d (k) - Cx d (k)) (3b) 

w(k) = H d x d (k)+K 2 [y(k) - Cx(k)) +K 2 d (y d (k) - Cx d (k)). (3c) 
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The dynamic of the estimation error is : 

e(fc + 1) = (A - Lc)e(k) + [a a - L d c)s d (k) + B5f k 
with 



(4) 



Sf k = f(Hx(k),H d x d (k)) - f[v(k),w(k)). 



From (35), we obtain 



\ 3 M\ - Tf 



(H - &C)e(k) - KJCe d (k) 

(H d - K 2 d C)e d (k) - K 2 Ce(k) 



(5) 



2.2 Observer synthesis conditions 

This subsection is devoted to the observer synthesis method that provides a sufficient 
condition ensuring the asymptotic convergence of the estimation error towards zero. The 
synthesis conditions, expressed in term of LMI, are given in the following theorem. 

Theorem 2.1. The estimation error is asymptotically stable if there exist a scalar a > and matrices 
P = P T > 0, Q = Q T > 0, R,R d ,K l ,K 2 ,K d and ¥? d of appropriate dimensions such that the 
following LMI is feasible : 



M i3 M 14 


m[ 5 


M[ 6 


-QM 23 M 24 


M 2 T 5 


Mj 6 


(*) M 33 








(*) w -p 








(*) W (*) 


-«7/^ 






(*) (*) (*) (*) -«t? 



< 



(6) 



where 



Mi 3 = 


A T PB- 


C T RB 


M 14 


= A T P- 


C r R 


M 15 = 


7/(«H- 


-tfc) 


JM 


16 = 7/* 


2 C 



M 23 = AJPB - C T R d B 

M 24 = AjP - C T R d 

M 25 = 7 2 K d C 

M 26 = 7 2 [aH d - K 2 C) 



M 



33 



iT 



PB - al a 



(7a) 
(7b) 
(7c) 
(7d) 
(7e) 
(7f) 
(7g) 
(7h) 
(7i) 
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The gains L and L d , K l , K 2 , K d and K 2 , are given respectively by 



L = p-!R r , L d 



P-^J 



K l 



K 



-K\ K 2 = -K 2 , 

a ix 



K], K 2 



-Ki 



Proof. Consider the following Lyapunov-Krasovskii functional : 

V k = e T (k)Pe(k) + '^(eJ(k)Qe,(k) 



Using the dynamics (4), we obtain 



V k+1 -V k = ^M^ k 



where 



Mi 



A T PA -P + Q A T PA d A T PB 
(*) AJPA d - Q AjPB 
(*) (*) B f PB 

$=[e T (k)eT(k)6fn, 

A = A-LC, 

A d = A d - L d C. 



(8) 



(9a) 

(9b) 
(9c) 
(9d) 



Using the notations K 1 = aK 1 , K 2 = txK 2 , K d = ocKl and K 2 = txK 2 ,, the condition (5) can be 
rewritten as follows : 



with 



M 2 



£jfM 2 & > o 



1 M, 



urn ° 





Ma 



-CClq 

rTi 



Mj 5 M 15 + Mf 6 M 16 M> M 25 + Mj J 6 M2 6 



and M 15 , Mjg, M 25 , M 2 6 are defined in (7). 
Consequently 

V k+ i-V k <Cl(Mi + M 2 )z k . 

By using the Schur lemma (see the Appendix), we deduce that the inequality 



(10) 

(11a) 
(lib) 

(12) 



Mi+M 2 < 
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is equivalent to 
where 



1M 4 



M 4 


< 






P+Q A T PB A T P 


M[ 5 


M« 


(*) -Q AjPB AjP 


Mf 5 


Mj 6 


(*) (*) M33 











(*) (*) w 


-p 








W (*) w 


(*) 


-«7/i, 





(*) (*) (*) 


(*) 


(*) 


-*7/4j 



(13) 



Using the notations R = L T P and R d = L^P, we deduce that the inequality M 4 < is 
identical to (6). This means that under the condition (6) of Theorem 2.1, the function V^ is 
strictly decreasing and therefore the estimation error is asymptotically stable. This ends the 
proof of Theorem 2.1. □ 

Remark 2.2. The Schur lemma and its application in the proof of Theorem 2.1 are detailed in the 
Appendix of this paper. 

2.3 Illustrative example 

In this section, we present a numerical example in order to valid the proposed results. 
Consider an example of an instable system under the form (1) described by the following 
parameters : 



420 
042 
03 



0.01 
0.01 




0.5 0.3 
0.5 0.3 
0.3 0.3 



, H = [1 1] , 



and 



where 



H d = [1 0] , C = [1 0] 
f(Hx,H d x d ,y) =<y f 



sin{x 1 {k)+x 3 {k)) 
cos{x 2 (k-l)) 



Ix 1 x 2 x 3 ] T 



and 7 j- = 10 is the Lipschitz constant of the function /. 

Applying the proposed method (condition (6)), we obtain the following gains : 

L = [0.0701 1.8682 2.9925] T , 



[0.3035 0.2942 0.0308] 
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K 1 = 0.9961, K 2 = -2.8074 x 10~ 5 , 

K\ = -9.0820 x 10~ 4 , K 2 d = -0.0075 
and 

OL = 10~ 7 . 

3. Extension to H^ performance analysis 

In this section, we propose an extension of the previous result to Hco robust observer design 
problem. In this case, we give an observer synthesis method which takes into account the 
noises affecting the system. 
Consider the disturbed system described by the equations : 

x(k + 1) = Ax(k) + A d x d (k) + E^cvik) + Bf(Hx(k),H d x d {k)) (14a) 

y(k) = Cx{k) + D w w{k) (14b) 

x{k) = x°(k), for k = -d,...,0 (14c) 

where u> (k) G £| is the vector of bounded disturbances. The matrices E u , and D w are constants 
with appropriate dimensions. 

The corresponding observer has the same structure as in (3). We recall it hereafter 
with some different notations. 



x(k + l) = Ax(k) + A d x d (k) + Bf(v 1 (k),v 2 (k)) 

+ L(y(k) - Cx(k)) + U (yc(k) - Cx d (k)) 



(15a) 



i h (k) = Hx{k) + K l (y{k) - Cx(k)) + K d (y d (k) - Cx d {k)) (15b) 

Mk) = H d x d (k)+K 2 [y(k) - CS(k)) +K 2 d (y d (k) - Cx d (k)). (15c) 

Our aim is to design the matrices L, L d , K 1 , K 2 , Kl and K 2 , such that (15) is an asymptotic 
observer for the system (14). The dynamics of the estimation error 

e(k) = x{k)-x{k) 

is given by the equation : 

e(k + 1) = (A - Lc)e(k) + U d - L d c)e d (k) + BSf k 

V ' V ' (16) 

E w - LD a ,)cv(k) - L d Da,cv d (k) 
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with 

Sf k = f(Hx(k),H d x d (k)) - f{v 1 (k),v 2 (k))) 

satisfies (5). 

re is to find the gains L,L d ,K ,K ,k, emu rv, 
converges robustly asymptotically to zero, i.e : 



The objective is to find the gains L,L d ,K ,K ,K\ and Kh such that the estimation error 



l|e||*3 < A||w||/. (17) 

where A > is the disturbance attenuation level to be minimized under some conditions that 

we will determined later. 

The inequality (17) is equivalent to 

l|e|l4<^(Nll| + ||w d |||- t w2 W) ? - ( 18 > 

Without loss of generality, we assume that 

w(k) = for k= -d,...,-\. 
Then, (18) becomes 

Mk< ^{\\"\\\ + \\"Al^ ■ (19) 

Remark 3.1. In fact, ifco(k) ^ for k = —d, ..., — 1, we must replace the inequality (17) by 

||e||/.<A(|M|i + i t ^(fiY (2°) 

z k=-d 

in order to obtain (19). 

Robust Hco observer design problem Li & Fu (1997) : Given the system (14) and the 
observer (15), then the problem of robust Hoo observer design is to determine the matrices 
L, L d , K 1 , K 2 , K] and Kj so that 

lim e(k) = for w{k) = 0; (21) 

k^co 

\\e\lq < X\\ca\\q Vw(fc) 7^ 0; e{k) = 0,k = -d,...,Q. (22) 

From the equivalence between (17) and (19), the problem of robust H^ observer design (see 
the Appendix) is reduced to find a Lyapunov function Vj. such that 

W, = AV + e T (k)e(k) - ¥-w T (k)a,(k) - ¥-w](k)cv d (k) < (23) 

where 

AV = V k+1 - V k . 

At this stage, we can state the following theorem, which provides a sufficient condition 
ensuring (23). 
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Theorem 3.2. The robust Hoo observer design problem corresponding to the system (14) and the 
observer (15) is solvable if there exist a scalar a > matrices P = P T > 0, Q = Q T > 0, 
R, Rj, K 1 , K 2 , K l d and K^ of appropriate dimensions so that the following convex optimization problem 
is feasible : 



where 



min(7) subject to T < 



" '-P+Q + k M13 







M M M[ 5 M[ 6 


(*) -QM 23 




Mj 4 Mj 5 Mj 6 


(*) (*) M 33 M 34 M 35 







(*) (*) (*) -7h 




E^P - C T R 


L w w (*) (*) -7k 




-Da.Rrf 




Mj 4 m£ m£ 


r 


"-P 
(*) -«7/J Sl 











Ej L ,P - C T R 




(*) (*) -«7/ J sj 




. -D u ,R d _ 









(24) 



(25) 



with 



M 34 
M 35 



B T P£a, - B T R T C, 
-B r RjD„, 



(26a) 
(26b) 



and M13, M u , M 15 , M 16 , M24, M 2 5, M 2 6, M 33 are de ned in (7). 

The gains L and E^,K l ,K 2 ,K l d ,K^ and the minimum disturbance attenuation level A are given 

respectively by 



K l 



K 



-'R T , 


u-- 


= p-'RJ 


a 


K 2 


Oi 




4 


- V 


A = 


^/2 ; > 





Proof. The proof of this theorem is an extension of that of Theorem 2.1. 

Let us consider the same Lyapunov-Krasovskii functional defined in (8). We show that if the 

convex optimization problem (24) is solvable, we have Wj- < 0. Using the dynamics (16), we 

obtain 

W k = n T S in 



where 



JM 



A T PE, 
AJPE, 
B T PE,„ -B 



In 
00 
00 

A T PD, 



AjPD a 
f PD„ 



iiT 



AjPE u , 
B f PE~ w 

PEl, - yl, 
DlPE' 



-AJPD a 
-B f PD a 

ElPDo 
DlPD' - 



lh 



(27) 



(28) 
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where 



Da, = L d D w 

[e T e] Sf k co T <vj] 



7 



A* 

2 ' 



(29a) 
(29b) 
(29c) 

(29d) 



The matrices Mj, A and A d are defined in (9). 

As in the proof of Theorem 2.1, since Sfa satisfies (5), we deduce, after multiplying by a scalar 

a > 0, that 



where 



S 2 



J/ J S 2 !? > 



-mM 3 



-oclc, 
00 
00 



(30) 



(31) 



and M3 is defined in (lib). 
The inequality (31) implies that 



W k = ti l (S 1 +S 2 )ri. 



(32) 

Now, using the Schur Lemma and the notations R = L T P and R^ = LjP, we deduce that 
the inequality Si + S2 < is equivalent to T < 0. The estimation error converges robustly 
asymptotically to zero with a minimum value of the disturbance attenuation level A = \/2j if 
the convex optimization problem (24) is solvable. This ends the proof of Theorem 3.2. □ 

Remark 3.3. We can obtain a synthesis condition which contains more degree of freedom than the 
LMI (6) by using a more general design of the observer. This new design of the observer can take the 
following structure : 



x{k + 1) = Ax(k) + A d x d (k) + Bf(v(k), w{k)\ 

+ L(y(k) - Cx(k)) + £L,(y l (k)-Cx l (k)) 



(33a) 



v{k) = Hx(k) + K 1 (y(k) - Cx(k)) + £ Kj (y,(k) - Cx { (k) 



(33b) 



(k) = H d x d (k) + K 2 (y(k) - Cm) + £K?(y,(fc) - Cx t (k) 



1=1 



(33c) 
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If such an observer is used, the adequate Lyapunov-Krasovskii/uwctona/ that we propose is under 
the following form : 

j=d i=j 

V k = e T (k)Pe(k) + £ £ Uj(k)Q, £i (k)) . (34) 

;=1,=1 V 

4. Systems with differentiable nonlinearities 

4.1 Reformulation of the problem 

In this section, we need to assume that the function / is differentiable with respect to x. 
Rewrite also / under the detailed form : 



f(Hx,H d z) 



■d,\n 



f x {R x x,H{z 



j q (H q x,H*z) 



(35) 



where H; e ]R S > X " and Hf e R r ' x " for all i e {l,...,q}. Here, we use the following 
reformulation of the Lipschitz condition : 



< aij < M(M < b,j < +00, v C e R s ', v z< e R r 



oo < 4 < 9 /' 



K 



y,C) < bfj < +oo, V C e R r ', V x' e R s 



(36) 
(37) 



where x' = H;x and z' = Hfz. 



The conditions (36)-(37) imply that the differentiable function / is 7 j--Lipschitz where 
\ 



V 



l='l 



' != s i 



^max £max | fl , ; |2, |b, ; |2 , ^ max (\a^M\^ , 

V;'=i /=i / 



^ 



The reformulation of the Lipschitz condition for differentiable functions as in (36) and (37) 
plays an important role on the feasibility of the synthesis conditions and avoids high gain as 
shown in Zemouche et al. (2008). In addition, it is shown in Alessandri (2004) that the use of 
the classical Lipschitz property leads to restrictive synthesis conditions. 

Remark 4.1. For simplicity of the presentation, we assume, without loss of generality, that f 
satis cs (36) and (37) with a,, = and af m = Ofor all i, I = 1, ..., q, j = 1, ..., s and m = 1, ..., r, where 
s = max (s,) andr = max (r,). Indeed, if there exist subsets S\,Sf C {l,...,q}, S2 C {l,...,s} and 

\<i<q \<i<q 

S2 C {1, ...,r} such that au ^ Ofor all (i,j) e S x x S2 and af m ^ Ofor all (l,m) G Sf x Sf, we can 
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consider the nonlinear function 



(38) 



f{xk,x k - d ) = f(Hx k ,HdX k -d)- { £ a,jHijHi)x k 

(!,/') 6 S]XS 2 

H a L H L H i) x k-d 

where 

H,j = e q (i)el(j) and H? m = e<,(l)ej t (m). 

Therefore, f satis es (36) and (37) with au = 0, af- = 0, £>,,- = bu — au and bf- = bf- — af,, and then 
we rewrite (la) as 

*k+i = Ax k + A d x k _ d + Bf(x k , x k _ d ) 

with 

A = A + B £ UijHijHi, A d = A d + B £ a fj H fj H i 

(i,j)eS 1 xS 2 {i,j)€S(xS* 

Inspired by Fan & Arcak (2003), we consider the following state observer : 

'=1 
*k+i = Ax k + A d x k _ d + V Be q {i)fi{v' k ,w l k ) 

,=1 (39a) 

+ L [y k - Cx k ) + L d [y k _ d - Cx k _ d ^J 

v' k = H,x k + K,[ Vk -Cx k ) (39b) 

w[ = H?x k _ d + Kf (y k _ d - Cx k -d) (39c) 

x k = x ,\Jk£ {-d,...,0} (39d) 

Therefore, the aim is to find the gains L e R" X P, L d G R" X P, K, G R S ' X P and Kf G R r ' x P, for 
i = 1, ..., q, such that the estimation error 

?k = *k - h (40) 

converges asymptotically towards zero. 

The dynamics of the estimation error is given by : 



E 

1=1 

where 



ejt+1 = (*- Lc)e k + (a a - L d c)e k _ d + £ Be q (i)Sf, (41) 

Sf i = fi(H i x h Hfx k )-f i (v k ,wi). 
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Using the DMVT-based approach given firstly in Zemouche et al. (2008), there exist Z; £ 
Co(HjX,v'),zf e CoiHfx^d,™ 1 ) for all i = \,...,q such that : 



l= s < 

E 



./='■ 



Sfi = E hij(k)el(j)xi + £ hfmelmf 



7=1 



where 



M = (H,-JC,c) £|t 
xi=(H?-Kfc)e k - d 

h,j(k) = ^ [ (z,(k),Hfx k _ d 



(42) 

(43) 
(44) 
(45) 

(46) 



Hence, the estimation error dynamics (41) becomes : 



£ Jt+l 



LC W 



L d C £jt _ 



+ E E M fc ) BH </*< 

i=\ ;=1 



(47) 



EE^-« B 

!= 1 ;=1 



///W 



4.2 New synthesis method 

The content of this section consists in a new observer synthesis method. A novel sufficient 
stability condition ensuring the asymptotic convergence of the estimation error towards zero 
is provided. This condition is expressed in term of LMI easily tractable. 

Theorem 4.2. The estimation error (40) converges asymptotically towards zero if there exist matrices 
P = P T > 0, Q = Q T > 0, R, R d , Kj and Kf, for i = l,...,q, of adequate dimensions so that the 
following LMI is feasible : 



-P + Q M A T P-C T R~ I 
(*) -Q N AJP-C T R a 
(*) (*) -Y YJP 

(*) (*) (*) -Y<< (E rf ) r P 
(*) (*) (*) (*) -P 



<0 



where 



M= [Mi(Ki)---M,(K,)] 
Mi(K,) = f (H, - K,C) T ...(H, - K,C) T ] 



(48) 

(49) 

(50) 
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JN 



N= [Ni(^)...N,(^)] 
(K?) = \ (Hf - K d C) T ...{H d - KfC) T ] 



r, times 



£ = B \Hu ■ ■ ■ H 1Sl H 2 \ ■ ■ ■ Hqsq 

Ed n Tjd Tjd TT TT 

— " ["11 ' ' ' "In "21 ■ ■ ■ H qr 

Y = diag(fi n I Sl ,...,fi lSl I Sl ,fi2ih 1 ,-,fiqs l] h 
Y d = diag(p d n I ri tirjru&lrz tfrfy) 



"~¥ Hii ~K 



(51) 
(52) 

(53) 
(54) 
(55) 
(56) 
(57) 



Hence, the gains L, L are given, respectively, by L = P R , L = P (R ) and the matrices 
K t , K.f are free solutions of the LMI (48). 

Proof. For the proof, we use the following Lyapunov-Rrasovskii functional candidate : 

i=d 

V k = e T k Pe k + £ e T k _,Qe k -, 
i=\ 

Considering the difference AV = V k+ \ — V k along the system (1), we have 



AV = eT 



A-LC) P(A-LC) -P + Q 



£ /c + £ k-d 



T 

A d -L d c) p[A d -L d c)-Q 



?k-d 



T T /' =c li =s i 

+ 2e T k ( A - LC) P[Ai- L d c) e k _ d + 2e T k ( A - Lc) P [£ BH^ 

\i'=l;'=l 



T i =c ll= r ' 

EE 



■2eI(A - LC)'P [ £ '£ B «?4 )+2el_ d (A d -L d CyP (£ ^BH^i, 



T 1= 11= s i 



,i=l ;=1 



. T l=c I)= r - 



'l=qj=s, 



l=q]=Si 



2el_ d (A d L d C ) p\EL BH ¥f, \ + \LL BH ^, ) p\LL BH -ik 



-j;»jj \ ' \ l—i l—i ~~~ij*i] / - \ i—i i—i -—ij-pi] 
1=1 ;=1 / \i=l/=l / \i=l/=l 



EE 

.=1;=1 



EE BH ^| P (EE B ^ 



i=qj=rt 
,,=1;=1 



where 



fci = Mk)xo ti = hUk)xf- 



(58) 
(59) 
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From (36) and (37), we have 



i=\ ;=1 \ n 'l D >1 , 



(60) 



Using (43) and (59), the inequalities (60) and (61) become, respectively, 
i'=l ;'=1 i=l ;'=1 ! i 



(61) 



(62) 



Consequently, 



where 



i=q]=r, 

EE« 

,=1 ;=1 



Hi? 
i 



Kfc) fLEk) r ^° 

i=l 7=1 6 z; 



AV < 



ft 



r n r 12 r 13 r 14 

(*) r 22 T23 r 2 4 

(*) (*) r 33 r 34 
.(*) W (*) r 44 . 






A-LC P A-LC -P+Q 



T 



r 13 = M T (ic 1 ,...,x ? 

Tl4 



A-LC) PE 

T 



A-LC J PE fl 
-L d c) T pfA rf -L d C 



L 23 



) T PE 



r 24 = N r (K 1 d ,... / ^) 



Q 



A d -L a C\ PE 



-Ti 



E T PE d 



T 33 = E I PE-Y 

r 34 

T 44 = (E fl ) J PE fl -Y fl 



■J) T PT, d - Y d 
tf = [(tfi) T (tU) T A&) T (&,) 



1 j 



(63) 

(64) 

(65) 
(66) 
(67) 
(68) 
(69) 
(70) 

(71) 

(72) 
(73) 
(74) 
(75) 

(76) 
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and M(Kj, ...,K q ), E, Y are defined in (49), (53) and (55) respectively. 

Using the Schur Lemma and the notation R = L T P, the inequality (48) is equivalent to 



<0. 



rii r 12 r 13 


r M " 


(*) r 22 T 2 3 


r 2 4 


(*) (*) r 33 


r 34 


(*) (*) (*) 


r 44 _ 



(77) 



Consequently we deduce that under the condition (48), the estimation error converges 
asymptotically towards zero. This ends the proof of Theorem 4.2. □ 

Remark 4.3. Note that we can consider a more general observer with more degree of freedoms as 
follows : 



i=q i=d 

h+\ = &h + ^ d x k _ d + £ Be q {i)fi{v\, w' k ) + £ L, ly k _, - Cx k _, 
1 = 1 /=o 

l=d , 

u k - HjX k + 2_, Ki,i [Vk-i - Cx k _i 

1=0 
l=d 

]d 



I'(- 



w 



- H i %-d + X] K i,l [Vk-d - Cx k _ d 
1=0 



(78a) 
(78b) 
(78c) 



This leads to a more general LMI using the general Lyapunov-Krasovskii functional : 

j=d i=j 
V k = e T k Pe k + ££ 4-iQfk-i 

j=\ ,=\ 

4.3 Numerical example 

Now, we present a numerical example to show the performances of the proposed method. We 
consider the modified chaotic system introduced in Cherrier et al. (2006), and described by : 



Gx + F{x(t),x(t-x)) 



(79) 



where 



-a 


a " 


1 


-1 1 





-P-tJ 



, F(x(t),x(t-r)) 



— aS tanh(xi(t)) 


esin(ux 1 (f — t)) 



Since the proposed method concerns discrete-time systems, then we consider the discrete-time 
version of (79) obtained from the Euler discretization with sampling period T = 0.01. Hence, 
we obtain a system under the form (la) with the following parameters : 



I 3 + TG, A rf = R3> 



'-aST 














eT 



and 



f(Xk,*k-d) 



tanh(.t 1 (fc)) 
sm(crxi(k — d) 
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that we can write under the form (35) with 

Hi = [1 0] , H( = [0 0] 

H 2 = [0 0] , R{ = [a 0] 

Assume that the first component of the state x is measured, i.e. : C = [l 0] . 
The system exhibits a chaotic behavior for the following numerical values : 

a = 9, fi = 14, 7 = 5, d = 2 

5 = 5, e = 1000, a = 100 

as can be shown in the figure 1. 
The bounds of the partial derivatives of / are 



100. 

50. 

0. 

-50- 




-10 -5 



Fig. 1. Phase plot of the system 



an = 1, 



1, a: 



21 



1, b a 2l = 1 















- 


-Tea 



According to the remark 4.1, we must solve the LMI (48) with 



b d 21 = b d 21 - a d 2l = 2, A d 



Hence, we obtain the following solutions : 

K x = 0.9999, K 2 = -0.0425, K d 
The simulation results are shown in figure 2. 



[ 1.3394 " 







4.9503 


, L d = 





40.8525 




-1000 



1.792 xl0~ 13 , ^ = 100 
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Fig. 2. Estimation error behavior 

5. Conclusion 

This chapter investigates the problem of observer design for a class of Lipschitz nonlinear 
time-delay systems in the discrete-time case. A new observer synthesis method is proposed, 
which leads to a less restrictive synthesis condition. Indeed, the obtained synthesis condition, 
expressed in term of LMI, contains more degree of freedom because of the general structure 
of the proposed observer. In order to take into account the noise (if it exists) which affects 
the considered system, a section is devoted to the study of H x robustness. A dilated LMI 
condition is established particularly for systems with differentiable nonlinearities. Numerical 
examples are given in order to show the effectiveness of the proposed results. 



A. Schur Lemma 

In this section, we recall the Schur lemma and how it is used in the proof of Theorem 2.1. 
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Lemma A.l. Boyd et al. (1994) Let Qi, Q 2 and Q3 be three matrices of appropriate dimensions such 
that Qi = Qj and Q3 = Qj . Then, the two following inequalities are equivalent : 



Qi Qi 
Q2Q3. 



<0, 



Q 3 < and Qj - Q 2 Q^Ql < 0. 



(80) 
(81) 



Now, we use the Lemma A.l to demonstrate the equivalence between Mi + M2 < and 
M 4 < 0. 
We have 



Mi +M 2 



A T PB 



-P+Q 

(*) -Q 

(*) (*) B T PB-u.I q 



A]PB 



A T PA A T PA d 0' 
(*) A T d PA d 

. (*) (*) 0. 



«7f 



M[ 5 M 15 + M[ 6 M 16 MJ-M25 + MLM 26 

T 



By isolating the matrix 



A 



0) 
0) 



P 

0-~2 



m[.m 26 + MJ 5 M 25 



w 







fl^L 



^7/^! 



f ls \ 



we obtain 



M X +M 2 



-P + Q A l PB 
(*) -Q AjPB 
(*) (*) B T PB-aI !j 



A> 


M[ 5 


MiV 


&l 


Mj 5 


Mj 6 












Y(-A) _1 Y 



Mis M 25 
M X6 M 26 



where 



By setting 



\P 


0" 


01* 








J.J 





"-P + Q A r PB 




~A T M[ 5 M[ 6 


Qi = 


(*) -Q AjPB 
(*) (*) B T PB-ocl q _ 


, Qi = 


k\ Mj 5 Mj 6 
_ 


have 










Mi +M 2 


= Qi-(. 


liQ^Ql- 



Y and Q 3 



-A 



(82) 



(83) 



(84) 



Since Q3 < 0, we deduce from the Lemma A.l that 

Mi + M 2 < 
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is equivalent to (80), which is equivalent to 

M 4 < 

where M4 is defined in (13). This ends the proof of equivalence between Mi + M2 < and 
M4 < 0. The Lemma A.l is used of the same manner in theorem 3.2. 

B. Some Details on Robust H OJ Observer Design Problem 

Hereafter, we show why the problem of robust H^ observer design is reduced to find a 
Lyapunov function 14 so that W^ < 0, where W^ is defined in (23). In other words, we show 
that Wj- < implies that the inequalities (21) and (22) are satisfied. 

If co(k) = 0, we have W^ < implies that AV < 0. Then, from the Lyapunov theory, we deduce 
that the estimation error converges asymptotically towards zero, and then we have (21). 
Now,i(co(k) ^ 0; e(k) = 0,k = -d, ...,0, we obtain W k < implies that 

N \2 N ,2 N N 

E ii £ « 11 2 < t E llw{k) l|2 + t E IK w l|2 ~ E ( Vk+i - Vk) (85) 

k=0 k=0 k=0 k=0 

Since without loss of generality, we have assumed that to(k) = for k = —d,...,—l and 
e(fc) = 0,fc = — d,...,0, we deduce that 

k=0 k=0 k=0 k=0 k=0 

When N tends toward infinity, we obtain 

co i2 oo i2 oo-rf i2 N \2 N-rf 

E l|e(*) ll 2 < t E ll«« II 2 + t E ll«( fc ) ll 2 ^ t E II ^ w ll 2 + t E II w « ll 2 - ( 87 ) 



As 



2 z_i " v '" 2 ^^ w " - 2 "" 2 

'— " J:=0 i=0 t=0 



co co— d 

Eii^wn 2 = E IkWII 2 = 11^111 



*=0 fc=0 

then the final relation (22) is inferred. 
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1. Introduction 

The integration of information from a combination of different types of observed 
instruments (sensors) are often used in the design of high-accuracy control systems. Typical 
applications that benefit from this use of multiple sensors include industrial tasks, military 
commands, mobile robot navigation, multi-target tracking, and aircraft navigation (see (hall, 
1992, Bar-Shalom, 1990, Bar-Shalom & Li, 1995, Zhu, 2002, Ren & Key, 1989) and references 
therein). One problem that arises from the use of multiple sensors is that if all local sensors 
observe the same target, the question then becomes how to effectively combine the 
corresponding local estimates. Several distributed fusion architectures have been discussed 
in (Alouani, 2005, Bar-Shalom & Campo, 1986, Bar-Shalom, 2006, Li et al., 2003, Berg & 
Durrant-Whyte, 1994, Hamshemipour et al., 1998) and algorithms for distributed estimation 
fusion have been developed in (Bar-Shalom & Campo, 1986, Chang et al., 1997, Chang et al, 
2002, Deng et al, 2005, Sun, 2004, Zhou et al., 2006, Zhu et al, 1999, Zhu et al, 2001, Roecker 
& McGillem, 1998, Shin et al, 2006). To this end, the Bar-Shalom and Campo fusion formula 
(Bar-Shalom & Campo, 1986) for two-sensor systems has been generalized for an arbitrary 
number of sensors in (Deng et al., 2005, Sun, 2004, Shin et al., 2007) The formula represents 
an optimal mean-square linear combination of the local estimates with matrix weights. The 
analogous formula for weighting an arbitrary number of local estimates using scalar weights 
has been proposed in (Shin et al., 2007, Sun & Deng, 2005, Lee & Shin 2007). 
However, because of lack of prior information, in general, the distributed filtering using the 
fusion formula is globally suboptimal compared with optimal centralized filtering (Chang et 
al, 1997). Nevertheless, in this case it has advantages of lower computational requirements, 
efficient communication costs, parallel implementation, and fault-tolerance (Chang et al., 
1997, Chang et al, 2002, Roecker & McGillem, 1998). Therefore, in spite of its limitations, the 
fusion formula has been widely used and is superior to the centralized filtering in real 
applications. 

The aforementioned papers have not focused on prediction problem, but most of them have 
considered only distributed filtering in multisensory continuous and discrete dynamic 
models. Direct generalization of the distributed fusion filtering algorithms to the prediction 
problem is impossible. The distributed prediction requires special algorithms one of which 
for discrete-time systems was presented in (Song et al. 2009). In this paper, we generalize the 
results of (Song et al. 2009) on mixed continuous-discrete systems. The continuous-discrete 
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approach allows system to avoid discretization by propagating the estimate and error 
covariance between observations in continuous time using an integration routine such as 
Runge-Kutta. This approach yields the optimal or suboptimal estimate continuously at all 
times, including times between the data arrival instants. One advantage of the continuous- 
discrete filter over the alternative approach using system discretization is that in the former, 
it is not necessary for the sample times to be equally spaced. This means that the cases of 
irregular and intermittent measurements are easy to handle. In the absensce of data the 
optimal prediction is given by performing only the time update portion of the algorithm. 
Thus, the primary aim of this paper is to propose two distributed fusion predictors using 
fusion formula with matrix weights, and analysis their statistical properties and relationship 
between them. Then, through a comparison with an optimal centralized predictor, 
performance of the novel predictors is evaluated. 

This chapter is organized as follows. In Section 2, we present the statement of the 
continuous-discrete prediction problem in a multisensor environment and give its optimal 
solution. In Section 3, we propose two fusion predictors, derived by using the fusion 
formula and establish the equivalence between them. Unbiased property of the fusion 
predictors is also proved. The performance of the proposed predictors is studied on 
examples in Section 4. Finally, concluding remarks are presented in Section 5. 

2. Statement of problem - centralized predictor 

We consider a linear system described by the stochastic differential equation 

x t =F t x t +G t v t , t>0, (1) 

where x t e iR n is the state, v t s 5R q is a zero-mean Gaussian white noise with covariance 

E(v t vJ) = Q t 5(t-s),and F t e 5R" X \ G t e$R" > "', and Q, e r». 

Suppose that overall discrete observations Y t e SR m at time instants t 1 ,t 2 ,... are composed 

of N observation subvectors (local sensors) y| ',..., y[ , i.e., 

\=[yf-yr T l T ' P) 

where y|' , i=l,...,N are determined by the equations 

y m=H«x t +w«, yM e 3T\ 
J k k k k J k 

y( N >=H< N >x t +w< N >, y( N > S 9r-, ^ 

7 'k 'k 'k k J k 

k=l,2,...; t k+1 >t k >t =0; m=m 1 +-+m N , 
where y® e « m ' is the local sensor observation, H^' s « nxm ' , and {w^e^ 1 "', k=l,2,...j 
are zero-mean white Gaussian sequences, ^v' 1 ) ~ N( R' 1 ' | i=l ... N ■ The distribution of the 

initial state x is Gaussian, x ~N(x ,P ), and x , v t , and jwj [, i = l,...,N are assumed 
mutually uncorrelated. 
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A problem associated with such systems is to find the distributed weighted fusion predictor 
x t+A , A>0 of the state x t+A based on overall current sensor observations 

X: = {\ \}> t a <...<t k <t<t + A, A>0. (4) 

2.1 The optimal centralized predictor 

The optimal centralized predictor is constructed by analogy with the continuous-discrete 
Kalman filter (Lewis, 1986, Gelb, 1974). In this case the prediction estimate x° p A and its error 
covariance P t ° A are determined by the combining of time update and observation update, 



i°P l =V v°P e 



t s x s , t k Ss<t+A, x s=tk -x tk , 
P° pt =FP s opt +P° pt R T +Q s , P° pt =P opt , 



(5) 



where the initial conditions represent filtering estimate of the state x° pt and its error 
covariance P° pt which are given by the continuous-discrete Kalman filter equations (Lewis, 
1986, Gelb, 1974): 



Time update between observations : 
C-°P t "=Fv° pt t <T<t x opt =v opt 

X T t T X T , t H STSt k , X T=tkj X tkj , 

popf _p popt' + popt pT + A popt _popt 

Observation update at time t k : 

x opt =x opt- +L opt/ Y _ H -opt"] 

'k 'k *k \ 'k tk *k / 

L opt =p opf H T /„ pOpt H T +R v "' 

tk t k t k ^ t k t k t k t k 

P°pt=fl -L° P 'H. )p opl ". 
tk \ t k w j t k 



(6a) 



(6b) 



Here I n 



the 
H (N) r 



nxn identity matrix, Q t =G t Q t GJ, 
, R tk =diag[R« 



Y t T = 



0) 



(N) 

y\J 



R\ ' , and the matrices F t , G t , Q t and Rj 1 ' 



are defined in (l)-(3). Note that in the absence of observation Y t , the centralized predictor 

includes two time update equations (5) and (6a), and in case of presence at time t=t k the 

initial conditions x° p and P° p for (5) computed by the observation update equations (6b). 

Many advanced systems now make use of a large number of sensors in practical 
applications ranging from aerospace and defence, robotics automation systems, to the 
monitoring and control of process generation plants. Recent developments in integrated 
sensor network systems have further motivated the search for decentralized signal 
processing algorithms. An important practical problem in the above systems is to find a 
fusion estimate to combine the information from various local estimates to produce a global 



42 Discrete Time Systems 

(fusion) estimate. Moreover, there are several limitations for the centralized estimators in 
practical implementation, such as computational cost and capacity of data transmission. 
Also numerical errors of the centralized estimator design are drastically increased with 
dimension of the state x. e *R n and overall observations Y, s s Jt m . In these cases the 
centralized estimators may be impractical. In next Section, we propose two new fusion 
predictors for multisensor mixed continuous-discrete linear systems (1), (3). 

3. Two distributed fusion predictors 

The derivation of the fusion predictors is based on the assumption that the overall 
observation vector Y t combines the local subvectors (individual sensors) yj ',..., y[ , which 
can be processed separately. According to (1) and (3), we have N unconnected dynamic 
subsystems ( i = 1,...,N ) with the common state x t and local sensor yj 1 ' : 

x t =F t x t +G t v t , t>t , 

yf=Hfx t +wf>, (7) 

k=l,2,...; t k+1 >t k >t =0, 

where i is the index of subsystem. Then by the analogy with the centralized prediction 
equations (5), (6) the optimal local predictor x® A based on the overall local observations 
jyj 1 ',... ,y[ l '\, t k <t<t + A satisfies the following time update and observation update 
equations: 

\ x^-Ex®, t k <s<t+A, x (i) t =x?>, 

J S S b K S -t k t k ;q\ 

P s (ii) =EP s (ii) +P s (ii) F s T +Q s , P (ii > =P (ii) , 

I s ss s s ^s' s=t k t k ' 

where the initial conditions x: and its error covariance P,' u ' are given by the continuous- 
discrete Kalman filter equations 

Time update between observations : 
xf=F T xf, t k . a <T<t k , *®^-x® , (9a) 

!>(")"=£ p( ii )" + p( ii )"pT + Q p(ii)' _p(ii) 

Observation update at time t k : 
x®-xf+L®(y®-H®#" 

h \ l k \ J t k l k [ k 

I® =P (U) ~Hf (Hf Pf y Hf +R« f , (%) 

r k r k r k \ r k r k r k r k J 

PfWl n -L«Hf)pf>". 

r k \ n r k C k / C k 

Thus from (8) we have N local filtering xj'^xji, and prediction x[!f A =x® t+A estimates, 
and corresponding error covariances Pj and P t + A for i=l,.„,N and t > t k . Using these 
values we propose two fusion prediction algorithms. 



Distributed Fusion Prediction for Mixed Continuous-Discrete Linear Systems 



43 



3.1 The fusion of local predictors (FLP Algorithm) 

The fusion predictor x t+A of the state x t+A based on the overall sensors (2), (3) is 
constructed from the local predictors xj!f A , i = 1,...,N by using the fusion formula (Zhou et 
al, 2006, Shin et al., 2006): 



N N 

JFLP = Y ,(i) iffl VJi) _ 

M+A Zv d t+A x t+A' Zj t+A 1 n 

i=l i=l 



In, 



(10) 



where a ( t +' A ,. 
square criterion, 



,( N ) 



are nxn time- varying matrix weights determined from the mean- 



J t+A _t 



,(i) „(i) 
1 t+A A t+A 



(11) 



-FLP 



The Theorems 1 and 2 completely define the fusion predictor x l+A and its overall error 

)FLP__./.-FLP -FLP\ -FLP »FLP 

X t+A X t+A" X t+A ■ 



• t~\ r 1_j1 / ~ r Li. ~* r Li. \ 

covanance P t+A =cov(x t+A ,x t+A ), x 

Theorem 1: Let x; + ' A ,...pq +A are the local predictors of an unknown state x 
. .,a[ +A satisfy the linear algebraic equations 



a. The weights aJ +A 



t+A' 

(i) 



t+A 



Then 



N 

1 

i=l 



Z,(i) TpW p< 
d t+A \_ L t+A l t+A 



M p(iN) 



0. ZaW A =I n , j=l,...,N-l; 



(12) 



The local covariance P t " A =cov(xJ!; A ,X(| A ), xfl A =x t+A -xil A satisfies (8) and local cross- 
covariance P^l = cov(xj ! l ' A/ x|{.' A ), i ^ j describes the time update and observation update 
equations: 



P®- = Ft p®- +P (ii) - F t + q t , P ® ki _ p ffl> , iki < T < tk , 

pf=(i n+ i«H«)rf)-(i n+ i«)H®) T , t=t k , 

pf=F s P®+P s (% s T + Q s , P®,-P®, t k <s<t + A; 

,FLP 



(13) 



The fusion error covariance P t+A is groen by 

N 



pFLP = Y a (i) p(ij) a (i)' 



L t+A 



a t+A J - t+A^t+A- 



(14) 



M=i 



Theorem 2: The local predictors xJ + ' A ,...,x[ +A and fusion predictor x t+A are unbiased, i.e., 

e(x?>)=E(x t ) and E(x* + L A P )=E(x t+A ) for <t < t+A . 

The proofs of Theorems 1 and 2 are given in Appendix. 

Thus the local predictors (8) and fusion equations (10)-(14) completely define the FLP 
algorithm. In particular case at N = 2, formulas (10)-(12) reduce to the Bar- Shalom and 
Campo formulas (Bar-Shalom & Campo, 1986): 
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-FLP (1) -.(1) (2) .(2) 

A t+A d t+A A t+A d t+A A t+A' 



a (l) = r P (22) p(21)ir p (ll) + p(22) p (12) p(21)T 1 ,.,-■. 

d t+A \_ L t+A _i t+A J \_ L t+A Ti t+A _i t+A _i t+A J ' \ 1D ) 

a P) =rP< n > p( 1 2)lrp(U) + p(22) p(12) p(21)T 1 
d t+A \_ L t+A _i t+A J \_ L t+A L t+A _i t+A _i t+A J • 

Further, in parallel with the FLP we offer the other algorithm for fusion prediction. 

3.2 The prediction of fusion filter (PFF Algorithm) 

This algorithm consists of two parts. The first part fuses the local filtering estimates 
x| ' , . . . ,x • ' . Using the fusion formula, we obtain the fusion filtering (FF) estimate 



N N 

FF 
X 

l k 

i=l i=l 



M«, Ib<H, ( 16 ) 



where the weights b[ ,...,b[ ' do not depend on lead A. 

In the second part we predict the fusion filtering estimate x, using the time update 
prediction equations. Then the fusion predictor x t+A and its error covariance 
P t +^=cov(xf^,xJ7 A ), xf+ A =x l+A -xf+ A satisfy the following equations: 

I xr= Fs xr, t k < s <t + A, x™<, 

| p p FF=FsP p FF+p p FFFj+ Q s/ pFFF=pFF _ 

Next Theorem completely defines the PFF algorithm. 

Theorem 3: Let x[ ',...,x; ' are the local filtering estimates of an unknown state x t . Then 

a. The weights b[',...,b[ ' satisfy the linear algebraic equations 

N N 

Zb«[p®-Pf ]=0, £b«=I n , j = l N-l; (18) 

i=l i=l 

b. The local covariance Pi u ' and cross-covariance Pj in (18) are determined by equations (9) and 
(13), respectively; 

c. The initial conditions x t and P t in (17) are determined by (16) and formula 



pFF = y b (i) p (ij) b (j) (19) 

t k Z_r t k t k t k ' v i 



respectively; 

d. Thefusit 

The proof of Tlieorem 3 is given in Appendix. 

3.3 The relationship between FLP and PFF 

Here we establish the relationship betwet 
determined by (10) and (16), respectively. 



d. The fusion predictor x^ in (17) is unbiased, i.e., E(xf + F A )=E(x t+A ) . 



Here we establish the relationship between the prediction fusion estimates x t+A and x t+A 
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Theorem 4: Let x v t +& and xf_JJ be the fusion prediction estimates determined by (10) and (16), 
respectively, and the local error covariances P™ , t k < s < t+A, i,j = 1,...,N are nonsingular. Then 

xf + L A P =xf + F A F for A>0. (20) 

The proof of Theorem 4 is given in Appendix. 

Remark 1 (Uniqueness solution): When the local prediction covariances P t + A , i,j=l,...,N are 
nonsingular, the quadratic optimization problem (11) has a unique solution, and the wights 
a[ + ' A ,...,aj +A are defined by the expressions (11). The same result is true for the covariance 
P® and the weights b^.-./bf (Zhu et al, 1999, Zhu, 2002). 

Remark 2 (Computational complexity): According to Theorem 4, both the predictors FLP and 
PFF are equivalent; however, from a computational point of view they are different. To 
predict the state x t+A using FLP we need to compute the matrix weights a{ +A ,...,a{ +A for 
each lead A>0. This contrasts with PFF, wherein the weights b; ,...,b: are computed 
only once, since they do not depend on the leads A . Therefore, FLP is deemed more 
complex than PFF, especially for large leads. 

Remark 3 (Real-time implementation): We may note that the local filter gains Ly, the error 
cross-covariances P( , P t + A , and the weights aJ+ A , bj 1 ' may be pre-computed, since they do 
not depend on the current observations yj , i = 1,...,N , but only on the noises statistics Q t 
and Rj , and system matrices F t , G t , HJ 1 ', which are part of the system model (1), (3). 
Thus, once the observation schedule has been settled, the real-time implementation of the 
fusion predictors FLP and PFF requires only the computation of the local estimates xy , 
x[+ A , i = l,...,N and final fusion predictors x t+A and x t+A . 

Remark 4 (Parallel implementation): The local estimates xf' , xf+ A , i = l,...,N are separated 
for different sensors. Therefore, they can be implemented in parallel for various types of 
observations yf', i=l,...,N. 

4. Examples 

4.1 The damper harmonic oscillator motion 

System model of the harmonic oscillator is considered in (Lewis, 1986). We have 





2 , | X t + 

M„ -2a 



v t , 0<t<£ , (21) 



where x t =[x lt x 2t ] / and x lt is position, x 2t is velocity, and v t is zero-mean white 
Gaussian noise with intensity q, E(v t v s )=q5 t _ s , x ~N(x ,P ) . Assume that the observation 
system contains N sensors which are observing the position Xj t . Then we have 

y( 1 )=H«x t +wf), 

J *k 'k r k Ik 

: (22) 

y< N )=H< N )x t W N >, 0=t <t a <t 2 <...<t*, 

J 'k t k t k t k U 1 z 

where H? -Tl Ol.and w, , ;=1,...,N are uncorrelated zero-mean white Gaussian noises 
with constant variances r UJ , respectively. 
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For model (21), (22), three predictors are applied: centralized predictor (CP) in (5), (6), FLP 
in (10) and PFF in (16), (17). The performance comparison of the fusion predictors for 
N = 2,3 was expressed in the terms of computation load (CPU time T CPU ) and MSEs, 



' i,t+A 



=E (Xi,t+A-Xi, t+ A) / where 



-CP 



-FLP 



M,t+A A i,t+A' A i,t+A 



Ht+A ' 



i=l, 2. The model parameters, 



noise statistics, initial conditions, and lead are taken to 



oo n =3, a=2.5, t 

lT 



,-(!) 



3, q=5, r w =3.0,r^=2.0, r w =1.0. 



(2) = 



.(3) = 



x o =[l0.0 0.0] , P =diag[0.5 0.5], 
A=0.1~0.5 (sec), ^-^=0.1. 



(23) 



Figs. 1 and 2 illustrate the MSEs for position (x a ) , Pj t+A , P 1 t+A , P : t+A , and analogously for 
velocity (x 2 ), P 2 t+A' E 2t+A' ^t+A a * N = 2,3 and lead A =0.2. The analysis of results in 
Figs. 1 and 2 show that the fusion predictors FLP and PFF have the same accuracy, i.e., 
P; t+A =P; t+A , and the MSEs of each predictor are reduced from N = 2 to N = 3 . The usage of 
three sensors allows to increase the accuracy of fusion predictors compared with the optimal 
CP for two sensors, i.e., F"£+a(N=3) = P£+ A (N=3) < P^ A (N=2). Moreover the differences 
between optimal P it+A and fusion MSEs P it+A , Pu+a are small, especially for steady-state 
regime. The results of numerical experiments on an Intel® Core 2 Duo with 2.6GHz CPU 
and 3G RAM are reported. The CPU time for CP, FLP, and PFF are represented in Table 1. 
We find that although P^f A and P^ are equal (see Theorem 4), the CPU time T^ for 



evaluation of the prediction 



is 4-5 times less than T c 



f ~FLP / THi J i-i- s nr tLL J 

ror x Lt+A ( i cpu *- 1 cpu 



) and 



this difference tends to increase with increasing the dimension of the state n or the number 
of sensors N. This is due to the fact that the PFF's weights bf do not depend on the leads 
A in contrast to the FLP's weights aJ+ A . Also, since CPU time difference between CP and 
PFF is negligible, PFF algorithm prefer to implement in real application rather than CP, 
especially for distributed system or sensor network. 
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Fig. 1. Position MSE comparison of three predictors at N = 2,3 and lead A =0.2. 
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Fig. 2. Velocity MSE comparison of three predictors at N = 2,3 and lead A =0.2. 



Number 
of sensors 


Lead A (sec) 


CPU time (sec) 


T CP 
1 CPU 


T FLP 
1 CPU 


T PFF 
1 CPU 


N = 2 


0.1 


0.172 


0.826 


0.185 


0.2 


0.298 


1.475 


0.310 


0.3 


0.384 


1.863 


0.405 


0.4 


0.500 


2.552 


0.550 


0.5 


0.656 


3.137 


0.691 


N = 3 


0.1 


0.187 


1.024 


0.200 


0.2 


0.305 


1.743 


0.340 


0.3 


0.452 


2.454 


0.471 


0.4 


0.602 


3.306 


0.621 


0.5 


0.754 


4.203 


0.776 



Table 1. Comparison of CPU time at N = 2, 3 and A = 0.1 ~ 0.5 



4.2 The water tank mixing system 

Consider the water tank system which accepts two types of different temperature of the 
water and throw off the mixed water simultaneously (Jannerup & Hendricks, 2006). This 
system is described by 



-0.0139 







-0.0277 
0.1667 








T 





x t + 


l 


0.1667 




l 



f>0, 



(24) 



where x t =[x 1 , x 2 , x 3t ] and x t t is water level, x 2 , is water temperature, x 3 t is sensor 



temperature, and 



is a white Gaussian noise with intensity q, E(v t v s ) = qS t _ s , 



x ~N(x ,P ) . The measurement model contains two sensors (N = 2) which sense water 
level. Then we have 
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3/tM 1 ° °]\ +w h' 0=to<t t <t 2 <...<t*, i-lA 



(25) 



where to; ' and to; are uncorrelated white Gaussian sequences with zero-mean and 
constant intensities r ' and r ' , respectively. 




Fig. 3. CP, FLP and PFF MSEs for water level at lead A = 0.2s 




Lead A (sec) 

Fig. 4. Computational time of water tank mixing system using 3 predictors at leads 
A = 0.05, 0.1,... ,0.5s. 



The parameters are subjected to Cj = 1, 



,«• 



2, 



,< 2 ) 



1, t =3, 



= [1 1 Of 



P = diag[0.7 0.7 0.1] , t k -t k . x =0.1 , A = 0.05 ~ 0.5 . Fig. 3 illustrates the MSEs of the water 
level p£f +A , P{^ A and P PFF A at lead A =0.2. As we can see in Fig. 3 the CP is better than the 
fusion predictors and the fusion MSEs for water level (x x ) of FLP and PFF are equal, i.e., 

pf? . < P, F f p . = P, PFF . • The CPU times for CP, FLP and PFF are represented in Fie. 4, where it 

M,£+A L l,f+A M,f+A r O ' 

is shown that FLP requires considerably more CPU time than PFF, but CPU time of PFF is 
similar to CP. 

Thus, from Examples 4.1 and 4.2 we can confirm that PFF is preferable to FLP in terms of 
computation efficiency. 
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5. Conclusions 

In this chapter, two fusion predictors (FLP and PFF) for mixed continuous-discrete linear 
systems in a multisensor environment are proposed. Both of these predictors are derived by 
using the optimal local Kalman estimators (filters and predictors) and fusion formula. The 
fusion predictors represent the optimal linear combination of an arbitrary number of local 
Kalman estimators and each is fused by the MSE criterion. Equivalence between the two 
fusion predictors is established. However, the PFF algorithm is found to more significantly 
reduce the computational complexity, due to the fact that the PFF's weights b? do not 
depend on the leads A > in contrast to the FLP's weights a$ A . 

Appendix 

Proof of Theorem 1 

(a), (c) Equation (12) and formula (14) immediately follow as a result of application of the 
general fusion formula [20] to the optimization problem (10), (11). 

(b) In the absence of observations differential equation for the local prediction error 
yq' = x T -x™ takes the form 

x?=x T -xf=F T xf+G T v T . (A.l) 

Then the prediction cross-covariance P T ™=E >q'x^ ) associated with the x£ and x^' 
satisfies the time update Lyapunov equation (see the first and third equations in (13)). At 
t=tu the local error xl can be written as 

x f) =x _xf>= Xt 4^L»r y «-H»xf>x»"-L«rHf»x [ +wf>-H?>xf 1 = (l n -L«H«)xf>"-L ( t > wf>. (A.2) 

'k «k * k t k t k t k \J t k t k t k J t k t k L t k t k t k t k t k J \ n t k t k ^ t k t k t k 

Given that random vectors \ w « and w ® are mutually uncorrelated at i ^ j , we obtain 

observation update equation (13) for pW'=E|x' 1 'x® ) • 

This completes the proof of Theorem 1. 
Proof of Theorem 2 

It is well known that the local Kalman filtering estimates X*. 1 ' are unbiased, i.e., E(x! 1 1 ')=E(x T ) 
or El x^ )=E(x T -x^ 1=0 at 0<x<t k . With this result we can prove unbiased property at 
t k < t < t+A . Using (8) we obtain 

x< i) =x T -x< i) =F T xf+G T v T , x (i) =x®, t k <x<t+A, (A.3) 

or 

AE(x«)=F T E(x«), E(x®. )-e(x®)-0, t k <x<t + A. (A.4) 

Differential equation (A.4) is homogeneous with zero initial condition therefore it has zero 
solution E(xf) = or E(xf)=E(x T ), t k <x<t+A. 

Since the local predictors x[!; A , i = l,...,N are unbiased, then we have 



N 

s 

1=1 



INI 

E(xf + L A P )=Ea« A E(x« A )= 



N 

a<'> 
a t+A 

1 



E(x t+A )=E(x t+A ). (A.5) 
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This completes the proof of Theorem 2. 
Proof of Theorem 3 

a., c. Equations (18) and (19) immediately follow from the general fusion formula for the 

filtering problem (Shin et al., 2006) 

b. Derivation of observation update equation (13) is given in Theorem 1. 

d. Unbiased property of the fusion estimate x t+A is proved by using the same method as in 

Theorem 2. 

This completes the proof of Theorem 3. 

Proof of Theorem 4 

By integrating (8) and (17), we get 

x« A =<t(t + A,t k )x«, i=l N, xf + F A F =<t(t + A,t k )x[ F , (A.6) 

where <£(t,s) is the transition matrix of (8) or (17). From (10) and (16), we obtain 



N N N 

X t+A 2 J a t+A X t+A 2 J a t+A yJ l t+A ' t ki X t k 2^ A t,t k ,A X t k ' 
i=l i=l i=l 



(A.7) 



x^=<I»(t + A,t k) x F J=E<I»(t + A,t k )b«x«=EB« t/A x«, 

i=l i=l 

where the new weights take the form: 

Ag k ^-ag A *(HA,t k ), B« k , A =<D(t + A,t k )b«. (A.8) 

Next using (12) and (18) we will derive equations for the new weights (A.8). Multiplying the 
first (N-l) homogeneous equations (18) on the left hand side and right hand side by the 
nonsingular matrices 0(t+A,tk) and 0(t+A,tk) T , respectively, and multiplying the last non- 
homogeneous equation (18) by <l>(t+A,tk) we obtain 



2>(t+A,t k )b«[P® -Pf Molt+A,^) 1 =0, j=l N-l; 

i=l 

N 

XO(t+A,t k )b«=<t.(t+A,t k ). 



Using notation for the difference 6P S ' = P S -P s v/e obtain equations for 



B 1 A , i = l,...,N such that 

t,t k ,a 

N N 

£B« k , A 5P« N >O(t + A,t k ) T =0, j=l N-l; YXl k , A =0(t + A,t k ). (A.10) 

i=l i=l 

Analogously after simple manipulations equation (12) takes the form 

N N 

^a« A (l>(t + A,t k )cI>( t+ A,t k )- 1 [p« A -P^]=XA[:l k , A cI>(t + A,t k )- 1 6P t ^=0, 

i=1 i_1 (All) 

N N v ; 

XaS A ®(t+A,t k )=XA» k , A =®(t + A,t k ). 

i=l i=l 
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N N 

ZAg i/A <I.(t + A,t k )" 1 8P^ ) =0, j = l N-l; £AC0 -*(t*A,t k ). (A.12) 

i=l i=l 

As we can see from (A.10) and (A.12) if the equality 

5P l f)<D(t + A / t k ) T =0(t + A,t k )- 1 5P^ (A.13) 

will be hold then the new weights Aj'j A and B™ . satisfy the identical equations. To 
show that let consider differential equation for the difference SP^ '=Pg"-P s ' ' . Using (13) 
we obtain the Lyapunov homogeneous matrix differential equation 

dt^^tP-F^^FP-Fr^F^-F^J^dF^+dF^J, t k <s <t+A, (A.14) 
which has the solution 

5P« A I )=0(t + A,t k )5P« N )<D(t + A,t k ) T . (A.15) 

By the nonsingular property of the transition matrix <J>(t+A,t k ) the equality (A.13) holds, 
then AJ'j , = B?i A , and finally using (A. 7) we get 



N N 

MUM' 



- FLP = y a (i) - (i) _V R (i) - (i) _ - PFF , A -, ^ 

X t+A 2-i A t,t k ,A X t k ~ 2-1° t,t k ,A\ ~ X t+A- (A.W) 

This completes the proof of Theorem 4. 



6. References 



Alouani, A. T. & Gray, J. E. (2005). Theory of distributed estimation using multiple 
asynchronous sensors, IEEE Transations on Aerospace and Electronic Systems, Vol. 41, 
No. 2, pp. 717-722. 

Bar-Shalom, Y. & Campo, L. (1986). The effect of the common process noise on the two- 
sensor fused track covariance, IEEE Transactions on Aerospace and Electronic Systems, 
Vol. 22, No. 6, pp. 803-805. 

Bar-Shalom, Y. (1990). Multitarget-multisensor tracking: advanced applications, Artech House, 
Norwood, MA. 

Bar-Shalom, Y. & Li, X. R. (1995). Multitarget-multisensor tracking: principles and techniques, 
YBS Publishing. 

Bar-Shalom, Y. (2006). On hierarchical tracking for the real world, IEEE Transactions on 
Aerospace and Electronic Systems, Vol. 42, No, 3, pp. 846-850. 

Berg, T. M. & Durrant-Whyte, H. F. (1994). General decentralized Kalman filter, Proceedings 
of American Control Conference, pp. 2273-2274, Maryland. 

Chang, K. C; Saha, R. K. & Bar-Shalom, Y. (1997). On Optimal track-to-track fusion, IEEE 
Transactions on Aerospace and Electronic Systems, Vol. 33, No. 4, pp. 1271-1275. 



52 Discrete Time Systems 

Chang, K. C; Tian, Z. & Saha, R. K. (2002). Performance evaluation of track fusion with 

information matrix filter, IEEE Transactions on Aerospace and Electronic Systems, Vol. 

38, No. 2, pp. 455-466. 
Deng, Z. L.; Gao, Y.; Mao, L. & Hao, G. (2005). New approach to information fusion steady- 
state Kalman filtering, Automatica, Vol. 41, No, 10, pp. 1695-1707. 
Gelb, A. (1974). Applied Optimal Estimation, MIT Press, Cambridge, MA. 
Hall, D. L. (1992). Mathematical techniques in multisensor data Fusion, Artech House, London. 
Hashemipour, H. R.; Roy, S. & Laub, A. J. (1998). Decentralized structures for parallel 

Kalman filtering, IEEE Transactions on Automatic Control, Vol. 33, No. 1, pp. 88-94. 
Jannerup, O. E. & Hendricks, E. (2006). Linear Control System Design, Technical University of 

Denmark. 
Lee, S. H. & Shin, V. (2007). Fusion Filters Weighted by Scalars and Matrices for Linear 

Systems, World Academy of Science, Engineering and Technology, Vol. 34, pp. 88-93. 
Lewis, F. L. (1986). Optimal Estimation with an Introduction to Stochastic Control Theory, John 

Wiley & Sons, New York. 
Li, X. R.; Zhu, Y. M.; Wang, J. & Han, C. (2003). Optimal Linear Estimation Fusion - Part I: 

Unified Fusion Rules, IEEE Transations on Information Theory, Vol. 49, No. 9, pp. 

2192-2208. 
Ren, C. L. & Kay, M. G (1989). Multisensor integration and fusion in intelligent systems, 

IEEE Transactions on Systems, Man, and Cybernetics, Vol. 19, No. 5, pp. 901-931. 
Roecker, J. A. & McGillem, C. D. (1998). Comparison of two-sensor tracking methods based 

on state vector fusion and measurement fusion, IEEE Transactions on Aerospace and 

Electronic Systems, Vol. 24, No. 4, pp. 447-449. 
Shin, V.; Lee, Y. & Choi, T. (2006). Generalized Millman's formula and its applications for 

estimation problems, Signal Processing, Vol. 86, No. 2, pp. 257-266. 
Shin, V.; Shevlyakov, G & Kim, K. S. (2007). A new fusion formula and its application to 

continuous-time linear systems with multisensor environment, Computational 

Statistics & Data Analysis, Vol. 52, No. 2, pp. 840-854. 
Song, H. R.; Joen, M. G.; Choi, T. S. & Shin, V. (2009). Two Fusion Predictors for Discrete- 
Time Linear Systems with Different Types of Observations, International Journal of 

Control, Automation, and Systems, Vol. 7, No. 4, pp. 651-658. 
Sun, S. L. (2004). Multi-sensor optimal information fusion Kalman filters with applications, 

Aerospace Science and Technology, Vol. 8, No. 1, pp. 57-62. 
Sun, S. L. & Deng, Z. L. (2005). Multi-sensor information fusion Kalman filter weighted by 

scalars for systems with colored measurement noises, Journal of Dynamic Systems, 

Measurement and Control, Vol. 127, No. 4, pp. 663-667. 
Zhou, J.; Zhu, Y.; You, Z. & Song, E. (2006). An efficient algorithm for optimal linear 

estimation fusion in distributed multisensor systems, IEEE Transactions on System, 

Man, Cybernetics, Vol. 36, No. 5, pp.1000-1009. 
Zhu,Y. M. & Li, X. R. (1999). Best linear unbiased estimation fusion, Proceeding of 

International Conference on Multisource-Multisensor Information Fusion, Sunnyvale, 

CA, pp. 1054-1061. 
Zhu,Y. M.; You, Z.; Zhao, J.; Zhang, K. & Li, X. R. (2001) The optimality for the distributed 

Kalman filtering fusion with feedback, Automaica, Vol. 37, No. 9, pp. 1489-1493. 
Zhu, Y. M. (2002) Multisensor decision and estimation fusion, Kluwer Academic, Boston. 



New Smoothers for Discrete-time 

Linear Stochastic Systems with 

Unknown Disturbances 

Akio Tanikawa 

Osaka Institute of Technology 
Japan 



1. Introduction 



We consider discrete-time linear stochastic systems with unknown inputs (or disturbances) 
and propose recursive algorithms for estimating states of these systems. If mathematical 
models derived by engineers are very accurate representations of real systems, we do not 
have to consider systems with unknown inputs. However, in practice, the models derived by 
engineers often contain modelling errors which greatly increase state estimation errors as if 
the models have unknown disturbances. 

The most frequently discussed problem on state estimation is the optimal filtering problem 
which investigates the optimal estimate of state X\ at time t or x t+ \ at time f + 1 with minimum 
variance based on the observation Yj of the outputs {j/o>l/i>' • • ,yt}, i-e., Yf = a{y s ,s = 
0, 1, ■ ■ ■ ,f} ( the smallest u-field generated by {yo/Vi/ ' ' ' >3/f} ( see e -g-/ Katayama (2000), 
Chapter 4)). It is well known that the standard Kalman filter is the optimal linear filter in 
the sense that it minimizes the mean-square error in an appropriate class of linear filters (see 
e.g., Kailath (1974), Kailath (1976), Kalman (1960), Kalman (1963) and Katayama (2000)). But 
we note that the Kalman filter can work well only if we have accurate mathematical modelling 
of the monitored systems. 

In order to develop reliable filtering algorithms which are robust with respect to unknown 
disturbances and modelling errors, many research papers have been published based on the 
disturbance decoupling principle. Pioneering works were done by Darouach et al. (Darouach; 
Zasadzinski; Bassang & Nowakowski (1995) and Darouach; Zasadzinski & Keller (1992)), 
Chang and Hsu (Chang & Hsu (1993)) and Hou and Miiller (Hou & Miiller (1993)). They 
utilized some transformations to make the original systems with unknown inputs into some 
singular systems without unknown inputs. The most important preceding study related to 
this paper was done by Chen and Patton (Chen & Patton (1996)). They proposed the simple 
and useful optimal filtering algorithm, ODDO (Optimal Disturbance Decoupling Observer), 
and showed its excellent simulation results. See also the papers such as Caliskan; Mukai; Katz 
& Tanikawa (2003), Hou & Miiller (1994), Hou & R. J. Patton (1998) and Sawada & Tanikawa 
(2002) and the book Chen & Patton (1999). Their algorithm recently has been modified by the 
author in Tanikawa (2006) (see Tanikawa & Sawada (2003) also). 

We here consider smoothing problems which allow us time-lags for computing estimates of 
the states. Namely, we try to find the optimal estimate %t-L/t °f the state Xt—t based on the 
observation Yf with L > 0. We often classify smoothing problems into the following three 
types. For the first problem, the fixed-point smoothing, we investigate the optimal estimate 
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tffc/f of the state Xj- for a fixed k based on the observations {Yf,f = fc + 1, fc + 2, ■ ■ ■ }. Algorithms 
for computing X/t/f, t = k + \,k + 2, ■ ■ ■ , recursively are called fixed-point smoothers. For 
the second problem, the fixed-interval smoothing, we investigate the optimal estimate x t / N 
of the state Xt at all times t = 0, 1, • • • , N based on the observation Y^ of all the outputs 
{yO/ yi/ ' ' ' / Vn } ■ Fixed-interval smoothers are algorithms for computing x t / n, t = 0, 1, ■ ■ ■ , N 
recursively. The third problem, the fixed-lag smoothing, is to investigate the optimal estimate 
Xt-L/t or me state Xt-L based on the observation Yf for a given L > 1. Fixed-lag smoothers 
are algorithms for computing x t _i/ t , t = L + 1, L + 2, • • ■ , recursively. See the references such 
as Anderson & Moore (1979), Bryson & Ho (1969), Kailath (1975) and Meditch (1973) for early 
research works on smoothers. More recent papers have been published based on different 
approaches such as stochastic realization theory (e.g., Badawi; Lindquist & Pavon (1979) and 
Faurre; Clerget & Germain (1979)), the complementary models (e.g., Ackner & Kailath (1989a), 
Ackner & Kailath (1989b), Bello; Willsky & Levy (1989), Bello; Willsky; Levy & Castanon (1986) 
Desai; Weinert & Yasypchuk (1983) and Weinert & Desai (1981)) and others. Nice surveys can 
be found in Kailath; Sayed & Hassibi (2000) and Katayama (2000). 

When stochastic systems contain unknown inputs explicitly, Tanikawa (Tanikawa (2006)) 
obtained a fixed-point smoother for the first problem. The second and the third problems 
were discussed in Tanikawa (2008). In this chapter, all three problems are discussed 
in a comrehensive and self-contained manner as much as possible. Namely, after some 
preliminary results in Section 2, we derive the fixed-point smoothing algorithm given in 
Tanikawa (2006) in Section 3 for the system with unknown inputs explicitly by applying the 
optimal filter with disturbance decoupling property obtained in Tanikawa & Sawada (2003). 
In Section 4, we construct the fixed-interval smoother given in Tanikawa (2008) from the 
fixed-point smoother obtained in Section 3. In Section 5, we construct the fixed-lag smoother 
given in Tanikawa (2008) from the optimal filter in Tanikawa & Sawada (2003). 
Finally, the new feature and advantages of the obtained results are summarized here. To the 
best of our knowledge, no attempt has been made to investigate optimal fixed-interval and 
fixed-lag smoothers for systems with unknown inputs explicitly (see the stochastic system 
given by (l)-(2)) before Tanikawa (2006) and Tanikawa (2008). Our smoothing algorithms have 
similar recursive forms to the standard optimal filter (i.e., the Kalman filter) and smoothers. 
Moreover, our algorithms reduce to those known smoothers derived from the Kalman filter 
(see e.g., Katayama (2000)) when the unknown inputs disappear. Thus, our algorithms are 
consistent with the known smoothing algorithms for systems without unknown inputs. 

2. Preliminaries 

Consider the following discrete-time linear stochastic system for t = 0, 1, 2, ■ ■ ■ : 

x t+ i = A t x t + B t u t + E t d t + It, (1) 

yt = Q x t + t] t , (2) 



where 



Xt G R" the state vector, 

yt € R" ! the output vector, 
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lit G R' the known input vector, 

dj £ R 1 ? the unknown input vector. 

Suppose that ft and t]t are independent zero mean white noise sequences with covariance 
matrices Qt and Rf. Let At, Bt, Ct and Ej be known matrices with appropriate dimensions. 
In Tanikawa & Sawada (2003), we considered the optimal estimate $t+l/t+\ of the state x t +\ 
which was proposed by Chen and Patton (Chen & Patton (1996) and Chen & Patton (1999)) 
with the following structure: 

Zf+1 = ff+i Zf + Tf+i B t u t + K t+1 y t , (3) 

x t+i/t+i = 2 f+i + H t+i Vt+li (4) 

for t = 0, 1,2, • • • . Here, fg/o i s chosen to be Zg for a fixed Zg. Denote the state estimation error 
and its covariance matrix respectively by ej and Pf. Namely, we use the notations e.% = Xf — Xf/t 
and Pt = E{gf ej } for t = 0, 1, 2, ■ ■ ■ . Here, E denotes expectation and T denotes transposition 
of a matrix. We assume in this paper that random variables eg, {?/(}, {£t } are independent. As 
in Chen & Patton (1996), Chen & Patton (1999) and Tanikawa & Sawada (2003), we consider 
state estimate (3)-(4) with the matrices F t+ i, T t+ i, Hf+l and ^f+1 of the forms: 

(5) 
(6) 

(7) 
H C t , (8) 

(9) 

The next lemma on equality (6) was obtained and used by Chen and Patton (Chen & Patton 
(1996) and Chen & Patton (1999)). Before stating it, we assume that Ej. is a full column rank 
matrix. Notice that this assumption is not an essential restriction. 

Lemma 2.1. Equality (6) holds if and only if 

rank (C t+ \ E t ) = rank (Ef ) . (10) 

When this condition holds true, matrix H t +\ which satisfies (6) must have the form 

H t+1 = E t {(Q + i E t ) T (Q + i Et)}' 1 (C f+1 E t f . (11) 

Hence, we have 

Q+i H i+ i = Q+i E t { (Q +1 E t ) T (Q +1 Et)}' 1 (Q +1 E t f (12) 

which is a non-negative definite symmetric matrix. I 



K t+1 = K t+1 + K t+1' 




Et = Ht+1 Q + i E f , 




Tf+i = I - Ht+i Q+i, 




ff+i = At — H t+ \ C t+ \ A t - 


-4 


K t+\ = F t+i H t- 
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When the matrix Kr, , has the form 

4ri = A\+\ (Pt Ct T - Ht Rt) (q P t C t T + R f ) _1 , (13) 

A) +l = A t -H t+l C l+1 A t , (14) 

we obtained the following result (Theorem 2.7 in Tanikawa & Sawada (2003)) on the optimal 
filtering algorithm. 

Proposition 2.2. If Ct H t and R t are commutative, i.e., 

Q H t R t = R t C t H t , (15) 

then the optimal gain matrix Kj +1 which makes the variance of the state estimation error e t+ \ minimum 
is determined by (13). Hence, we obtain the optimal filtering algorithm: 

xt+l/t+l = A )+i {xt/t + Gt (yt - Q x t/t )} + fff+i y t +i + T t+1 B t u t , (16) 

P m = A] +1 M, A) +l T + T t+ i Qt T t+1 T + H t+l R l+1 H t+l T , (17) 



where 



and 



G t = [PtQ 1 -H t R t )[CtP t Ct 1 +R t ) \ (18) 



M t = P,-Gt[CtP,-RtHt T ). (19) 



Remark 2.3. If the matrix Rt has the form 

Rt = r,l 

with some positive number rt for each t = 1, 2, • • • , then it is obvious to see that condition (15) 
holds. | 

Finally, we have the following proposition which indicates that the standard Kalman filter is 
a special case of the optimal filter proposed in this section (see e.g., Theorem 5.2 (page 90) in 
Katayama (2000)). 

Proposition 2.4. Suppose that Et = O holds for all t (i.e., the unknown input term is zero). Then, 
Lemma 2.1 cannot be applied directly. But, we can choose Ht = O for all t in this case, and the optimal 
filter given in Proposition 2.2 reduces to the standard Kalman filter. I 

3. The fixed-point smoothing 

Let k be a fixed time. We study an iterative algorithm to compute the optimal estimate f jt/f or 
the state x/ c based on the observation Yj,f = k + l,k + 2, ■ ■ • , with Y t = cr{y s , s = 0, 1, ■ • ■ , t }. 
We define state vectors 6t, t = k, k + 1, ■ ■ ■ , by 



7 f+i = u t 



6 t , t = k,k + !,-■■ ; 6 k = x k . (20) 
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It is easy to observe that the optimal estimate §t/t of the state Qt based on the observation Yf 
is identical to the optimal smoother x^/ t in view of the equalities Qt = x k, t = k,k + 1,- ■ ■ . 
In order to derive the optimal fixed-point smoother, we consider the following augmented 
system for t = k, k + 1, • • • : 



x t+l 
Vt+i = 


A t O 
O I 

Q+i o) 


Xt + 
Qt 

~x t +\ 


Bt 

O "' + 

+ 7t+i- 





dt + 


I 
O 



It. 



Denote these equations respectively by 



x t+1 = A t x t + B t lit + E t d t + Jt Z,t, 
Vt+i = Q+i *i+\ + *lt+\> 



(21) 
(22) 



(23) 
(24) 



where 



xt 



' x t ' 

A. 


, A t = 


'A t O' 
I 


,B t = 


'B t ' 



, E t = 


'E,' 



,h = 


' i' 
o 



and C t+ i = [Q+i O] . 

Here, I and O are the identity matrix and the zero matrix respectively with appropriate 
dimensions. By making use of the notations 



Ht+i 
O 



Ht+i 
we have the equalities: 

Q+i Et = C t+ i Ef, T t+ i 



, T, 



t+i 



T t+ iO 

O I 



I O 

O I 



- H t+ i C 



+i w+i/ 



A] +1 = T w At 



1 t+i u 
o I 



,(U) p(l,2)~ 
( 1 t 


r 




. 






, 


-{ 


^t - 


- Xt/t 




xt - 


- x t/t 


,(2,1) „(2,2) 
f 1 f 


L««- 


- @t/t . 




Jt- 


- @t/t . 



We introduce the covariance matrix P( of the state estimation error of the augmented system 
(23)-(24): 

(25) 

Notice that P} ' is equal to Pf. Applying the optimal filter given in Proposition 2.2 to the 
augmented system (21)-(22), we obtain the following optimal fixed-point smoother. 

Theorem 3.1. IfCt Ht and Rt are commutative, i.e., 

C t H t R t = R t C t H t , (26) 

then we have the optimal fixed-point smoother for (21)-(22) asfolloius: 
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(i) the fixed-point smoother 

Xk/t+i = h/t + D f 00 [yt - Q f f/f ] , 

(ii) the gain matrix 

D t (fc)=P f (2 ' 1) Q T (c t P t C t T + R 
(Hi) the covariance matrix of the mean-square error 



d(2,1) 
i+1 



,(2,1) u(2,l) r T 



(C t P t -R t H, T ) \A) +1 T , 



(2,2) _ „(2,2) 1,(2,1) „T 



J f+1 2 t 



P^Q' QPfC^+Rf QP^ 



,(2A) ' 



(27) 
(28) 

(29) 
(30) 



(2 1) (2 2) 

Here, we note that P> ' ' = P> ' ' = Pj.. We nof/ce fftaf x t / f fs f/te optimal filter of the original system 
(l)-(2) given in Tanikawa & Sawada (2003). 

Proof Applying the optimal filter given by (16)-(17) in Proposition (2.2) to the augmented 
system (23)-(24), we have 



x t+i/t+i = A t+\ |^f/t + G t [y f - C t x t / t J | + H t+ i y t+ i + T t+1 B t u t . 
This can be rewritten as 



(31) 



x t+\/t+\ 



f+i/f+i 



A} +l O 
O I 



xj/t 
8t/t 



+ 



P t (U) C t T - H t R t 

- p (2,l) Q T 



x (QP t Q T + R t ) \y t -C t x t/t ) [ + 



H t+\ Vt+i 
O 



+ 



Tf + i B t u t 
O 



Thus, we have 



,(U) r T 



-1 



= A\ +l \ x t/t + \P™ C, 1 - H,R t ) (Q Pt C, J + R t ) (yt - C t x t /t] 
+Ht+i yt+i + Pf+i B t ut 



,(2,1) r J (r. n.rJ 



and 

Ot+vt+i = § t/t ^ 

Here, we used the equalities 



CtPtCt +R t = [Q O] 



CtPtCt 1 +R 



p(U) p (l,2) 

pW) p(2,2) 
L J t 1 1 

T 



o 



Rt 



(32) 
(33) 



(34) 
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and 



G t = [Ft 



o 



H t R t ) [CtPtQ +R, 



-1 



p(U) pU,2) 

p(2,l) p (2,2) 

p\ l,V) C t T - H t R t 

p(2,l) Q T 



C, 
O 



ri 



O 



QPtQ +R, 



-1 



QPtCj + R, 



(35) 



Thus, equalities (27)-(28) can be obtained from (33) due to 8 t /t = fjt/f- 

By using the notation Mt for the augmented system (23)-(24)which corresponds to the matrix 

Mf in Proposition (2.2), we have 



M t 



M?V M t (1 ' 2) 

mP a) m| 2 ' 2) 

= Pt-Gt(c t Pt-Rt [H t T o] 

(1-1) n(l-2) 



pv^v pv 



L^ 



(24) „(2,2) 



[Q O] 



P ( (1 ' 1) Q T -H t R t 
p(2,D Q r 

p(U) p(U) 

pW) pW) 



QP f Q J +R f 

[r,H ( t O] 



Thus, we have 



,(l,l) 



-,(U) 



,(1,1) r.T 



M y L > = P^>- [P^'C, 1 -H t Rt\ [C t PtCt T + Rt) [CtP^'-RtHt* 



M (i,2) = p(l,2) 



,(1,1) r.T 



Q 1 -H t Rt) [C,P t C 



qp; 



(u) 

1,2) 



m; 



(2,1) 



-,(2,1) -(2,1) r T 



- Pf> C t M Q P t C t ' +R t ) [C t Pi 1 ' 1 ' - R t H 



-l 



,(i-i) 



T 



M 



(36) 
(37) 
(38) 



and 



M] 



(2,2) 



f 2) -p\ 2 - l) Ct T \CtPtCt 



-l 



qp; 



(1,2) 



It follows from (17) in Proposition 2.2 that 



(39) 



Pt+i = A} +1 M t A} +1 + T t+ i h+i Qt+i h+i T M + H t+1 R t+1 H t+1 



r a 1 
o 


o~ 

I 




m; 


U) M (1,2)- 
2,D M (2,2)_ 







T 




+ 




) 


i 




' I" 




Qt+ 


l[ 


fo] 


o 


l 



+ 



Ht+i 
O 



R t +i [H t+1 T O] 



(40) 
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Equalities (29)-(30) follow from (38)-(40). Finally, we have equalities pf ,X) = pf ,2) = P^' 1] = 
Pk by the definition of P^. | 

We thus have derived the fixed-point smoothing algorithm for the state-space model which 
explicitly contains the unknown inputs. We can indicate that the algorithm has a rather simple 
form and also has consistency with both the Kalman filter and the standard optimal smoother 
derived from the Kalman filter as shown in the following remark. 

Remark 3.2. Suppose that Ef = O holds for all t (i.e., the unknown input term is zero) and 
that Ht = O for all f (as in Proposition 2.4). In this case, it follows from Theorem 3.1 that 

*t+i/t+i = At{*t / t + PtCt T (c t PtCt T + Rt) 1 (2/t-C t X i /t)\+B t Ht, (41) 



e 



t+i/t+i = 9t/t + p \ 2,l) Ct T (Q P t C t T + R t ) J (y t - Q x t/t ) , (42) 

p (2,i) = | p (2,i) _ p ( 2 ,i) qT r Q Pt c r + R \ - 1 Q Pt | A r > (43) 



and 

p (2,2) = p (2,2) _ p (2,l) qT ^ pt C T + ^ "I Q p (2,D\ ^ 

Here, we note that the state estimate %t+\lt+\ reduces to the state estimate Xf+i/f in Katayama 
(2000) when Hf = O holds. Moreover, Equalities (37)-(40) with the state estimates Xt+i/t+i 
and x t / t replaced respectively by $t+l/t anc ^ %t/t-l are identical to those for the pair of the 
standard Kalman filter and the optimal fixed-point smoother in Katayama (2000). Thus, it has 
been shown that this algorithm reduces to the well known optimal smoother derived from 
the Kalman filter when the unknown inputs disappear. This indicates that our smoothing 
algorithm is a natural extension of the standard optimal smoother to linear systems possibly 
with unknown inputs. I 

Let us introduce some notations: 

vt = yt- Q Xt/t, (45) 

L t = A} +1 (I-G t C t ), (46) 

f Lf-iLt-i •••L T , t > t 
W(t,r) = { (47) 

where the matrix Gt was defined by (18), i.e., 

G t = (P t C t T - H t Rt) (C t Pt C t T + Rt] ^ . (48) 



We then have the following results due to (27). 
Corollary 3.3. We have the equalities: 

h/t+i = *m + E D <( fc K = %* + p k E Y(i,fc) T Q T (CjPjC, 7 + Rt) Vi . (49) 

i=k i=k 
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Proof It is straightforward to show the first equality from (27). For the second equality, it is 
sufficient to prove the equality 

D t (k)=P k Y(t,k) T C t T (QPtCj + RtY 1 (50) 



for t > k. By virtue of (46), equality (29) can be rewritten as 

Pp) = p£? (i - Q_/ G, V) A? T = Pg> !,_/. (51) 

By using this equality recursively, we have 

p(2,l) _ (2,1) r T r T _ _ „(2,1) j T r T r T 

l t — h-2 L t-2 L t-1 — — L k L k L k+1 ■ ■ ■ L t-1 

= P k Y(t,kf. (52) 

Substituting this equality into (28), we obtain 

D t (k)=P k Y(t,k) T C t T (CtP t Ct T + R t y\ (53) 

i.e., (50). I 

Finally, we study the reduction of the estimation error by the fixed-point smoothing over the 
optimal filtering. Due to (27), we have 

pf' 2) = E[(x k -x k/t )(x k -x k/t ) T }. (54) 

Denote this matrix simply by P k /t- It then follows from (30) that 

Pk/,+1 = Pk/t - P f (2 ' 1} Q T (c t P t C t T + R t ) _1 C f P ( (2 ' 1)T . (55) 

Summing up these equalities for t = k, k + 1, • • • , s, we have 

Pm - Pk/s+i = t ^ (2 ' 1) Q T (qp,c, T + R t ) ^ c,p< 2 ' 1)r . (56) 

i=k 

Thus, the right hand side indicates the amount of the reduction of the estimation error by the 
fixed-point smoothing over the optimal filtering. 

4. The fixed-interval smoothing 

We consider the fixed-interval smoothing problem in this section. Namely, we investigate the 
optimal estimate X-t/N OI me state Xt at all times t = 0, 1, • • • , N based on the observation Y^ of 
all the states {yo,y\, ■ • ■ ,})n}- Applying equality (49), we easily obtain the following equality. 

Lemma 4.1. The equality 

x t /N = xt/t+i + Pt U J P t+ i~ l (Xf+i/N - *f+i/f+i) (57) 
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holds fort = 0,1 / ---,N-1. I 
Proof Using the notation 

v, = C, T (CiPjtf+R,) ~ l v it (58) 



we have 

h/i + i = h/k + PkEV(i,k) T v~i (59) 

i=k 

for k < t due to (49). In view of (59) , we also have 

f f 

x k /t+i = x k/k + P k v k + P k E Y(i,k) T v,=x k/k+1 + P k £ Y(U) T i7; (60) 

i=k+l i= k +l 

for k + 1 < t. Putting t + 1 = N and k = t + 1 in equality (59), we have 

N-l 

*t+lAN = *f+i/t+i + Pt+1 E Y(i,f + l) T i/-. (61) 

/=f+i 

Putting f + 1 = N and A: = t in equality (60), we have 

N-l N-l 

£ t/N = x t/t+1 + P t E Y(i,t) T Vi = x t/t+x + P t L t T E Y(f,f + l) T v--. (62) 

i=f+l i=t+l 

Substituting (61) into (62), we have 

*f/N = x t/t+l +PtU P t+ \ ($t+l/N - %t+l/t+l) ■ 

The above derivation is valid for t = 0, 1, • • • , N — 2. It is easy to observe that equality (57) 
also holds for t = N — 1. | 

It is a simple task to obtain the following Fraser-type algorithm from (57). 

Theorem 4.2. We obtain the fixed-interval smoother 

Xt/N = x t/t+i + Pt U T A t+1 , (63) 

A t = L f T A t+1 + Q T (c t P t C t T +R t ) ~\ t . (64) 

for t = N — 1, N — 2, • • • ,1,0. Here, we have \ N = 0. I 

Proof For t = 0, 1, ■ ■ ■ , N, we put 

A t = P f _1 (%n - %t) ■ (65) 

We then have Ajv = 0. Substituting (65) into (57), we obtain equality (63). Then, by utilizing 
(63) and (65), we have 

A, = Pr 1 (% f +l + p f L t T A t+i - x,/ t ) ■ (66) 

In view of the equality 

x i/t+i - x t/t = Pt vt (67) 
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which follows from (27) in Tanikawa & Sawada (2003), we obtain 

A t = L, T A t+1 + i7 t 

= L t T A, +1 + Q r (CtPtCt T +R t Y\ t . 



(68) 



Thus, we proved (64). I 

Remark 4.3. When £( = O holds for all t (i.e., the unknown input term is zero), we shall see 
that fixed-interval smoother (63)-(64) is identical to the fixed-interval smoother obtained from 
the standard Kalman filter (see e.g., Katayama (2000)). Thus, our algorithm is consistent with 
the known fixed-interval smoothing algorithm for systems without unknown inputs. This 
can be shown as follows. Assuming that £f = O, we have Ht = O for t = 0, 1, ■ ■ ■ , N (see 
Propositin 2.4). Note that in (59), i.e., 

t 
4/H-i = x k/k + P k J^ Y(i,k) T Vi 



X-k/t+i an d £jt/)c respectively reduce to x k / t and x k / k -\ which are respectively the optimal 
smoother and the optimal filter obtained from the standard Kalman filter. Then, the above 
equality is identical to (7.18) in Katayama (2000). Since the rest of the proof can be done in the 
same way as in Katayama (2000), we obtain the same smoother. I 

5. The fixed-lag smoothing 

We study the fixed-lag smoothing problem in this section. For a fixed L > 0, we investigate 
an iterative algorithm to compute the optimal state estimate x t _i/ t of the state %t-L based on 
the observation Y(. 
We consider the following augmented system: 



*f+l 




Xt 


= 


.Xt-L+l. 





A t O 
I O 

o 



... o 
... o 

I o 

O] 





Xt 




Bt 




it 




I 




X,-\ 


+ 


O 


u t + 


O 


d t + 


O 




.X t -L. 




O 




O 




O 



u. 



x t+ i 

Xt 



+ »/f+l- 



yt+i = [Q+i o 

.Xt-L+l. 

Denote these equations respectively by 

x^i = A t x t + B t lit + E t d t + J t It, 

yt+i = Q+i ^t+i + fjt+i, 



(69) 



(70) 



(71) 
(72) 
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where 



X{ 



Xt-l 

X t -L 
I 

O 



A t O ... O] 




"B t l 




"EH 


I o ... o 


, B t = 





, E t = 


O 


O I o_ 




_0_ 




.0. 



o 



and C f+1 = [C f+ i O ... O] 



Here, I and O are the identity matrix and the zero matrix respectively with appropriate 
dimensions. By making use of the notations 



H, 



f+i 



H, 



f+l 
O 

O 



and T/ 



f+l 



J _ Wf+i Q + i, 



we have the equalities: 



Q+i E f = [Q+i O ... O] 



O 



Q+i Ef, 



'f+l 



I- 



Ht+i 

O 



o 



[Q+i O ... O] 



T t+1 . 
O J . 


. O" 

. o 


O . 


. I _ 



A) +l = T t+1 A t 



T t +i O . 
O I . 


. O" 

. o 




o o . 


. I_ 





A t ...O" 




I O ... O 


__ 


O I o. 





Aj +1 0...0 

I o ... o 



o 



I o 



We introduce the covariance matrix Pj of the state estimation error of augmented system 
(71)-(72): 



P t = E< 



*< - *f/f 
xt-i-£t-i/t 



Xt-L-Xt-L/t 



Xt - *f/f 
*t-l-*t-l/t 



Xt-L-Xt-L/t 



(73) 
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By using the notations 



Pt-i,t-j/t — E <, {xt-i — $t-i/t) \x t _j — x t _j/ t 
Pt-i/t = Pt-i,l-i/t> 



we can write 



Pt/t Pv-yt 
Pt-l,t/t Pt-l/t 



P t-L,t/t Pf-L,t-l/t 



n,t-L/t 

°t-l,t-L/t 



"t-L/t 



(74) 



Here, it is easy to observe that P t / t = Pt holds. We also note that 

C t P t Ct T + R t = CtP t/t C t T + R t . 
From now on, we use the following notation for brevity: 

C t :=C t PtC f T + R t . 
Applying the optimal filter given in Proposition 2.2 to augmented system (71)-(72), we have 

xt+vt+i = A )+i {x7ft + Gt (y t - Q xjft) } + Ht+i Vt+i + T t+i &t u t , (77) 



(75) 



(76) 



where 



PtQ -H t Rt) [QPtCt +R 



-l 



P t/t C t l -H t R t 



t-i,t/t 



t-L,t/t 



Q> 



Ct 1 



cf 



(78) 



Identifying the component matrices of (77)-(78), we have the following optimal fixed-lag 
smoother. 



Theorem 5.1. IfQ Ht and Rt are commutative, i.e., 

Q H t R t = R t C t H t , 

then we have the optimal fixed-lag smoother for (l)-(2) as follows: 
(i) the fixed-lag smoother 



(79) 



(80) 



xt-j/t+i = xt-j/t + S t (j) (y t - C t x t /t) (/ = 0,1,--- ,L-1) , 

(ii) the optimal filter 

xt+yt+i = A )+\ {x t /t + Gt (yt - Q xt/t) } + Ht+ryt+i + T t+1 B t u t , (81) 

with Gt defined by (18) in Proposition 2.2, 
(Hi) the gain matrices 



St(j)= {Pt-j,t/tCt l -SojHtRt) Q 



-i 



(j = 0,l,..-,L-l), (82) 
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where Su stands for the Kronecker's delta, i.e. 

S: , -- 



1 for i = j 
for i^j ' 



(iv) the covariance matrix of the mean-square error 



f+i/f+i 



A} +1 M\°' 0) AJ +1 T + T t+1 Q t T t+1 ' 



Pt+u-j/t+i = 4+1 m\ 



(0,1) 



t-j,t+i/t+i 



t-i,t-j/t+l 



M 



t+l,t-j/t+l 
(W) 



and 



M 



hi) 



- t-i,t-j/t 



t-tt/t^t 



Ht+l R-t+1 H t+ i , 

(j = 0,l,...,L-l), 

(j = 0,l,---,L-l), 

(i,j = 0,1,... ,L-1), 



S 0ii H t R t )Ct 



QP, 



t l t,t-j/t - °o,j lx t m 



SmRtH 



(83) 

(84) 
(85) 
(86) 
(87) 



(i,j = 0,1,... ,L). (88) 



Remark 5.2. Since the equalities 



and 



>t/t 



f (0,0) 



Pt ( in Proposition 2.2 ) 



Mf'> = M t ( in Proposition 2.2 ) 

hold, the part of the optimal filter in Theorem 5.1 is identical to that in Proposition 2.2. When 
Et = O holds for all t (i.e., the unknown input term is zero), we shall see that fixed-lag 
smoother (80)-(88) is identical to the well known fixed-lag smoother (see e.g. Katayama (2000)) 
obtained from the standard Kalman filter. Thus, our algorithm is consistent with the known 
fixed-lag smoothing algorithm for systems without unknown inputs. This can be readily 
shown as in Remark 4.3. | 

Proof of Theorem 5.1 Rewriting (77)-(78) with the component matrices explicitly, we have 



- T f+l/f+l 

%M-1 
*f-l/f+l 



L x f-L+l/f+l-l 



A )+i {*t/t + (Pt/tQ T - H t R t ) Cr 1 {y t - Qfj/t)} 

*t/t + ( p t/t Ct T - H t R t J Zr l {y t - Q x t/t ) 

*t-i/t + p t-u/t Ct T Ct~ {yt - Q x t/t ) 



x t-L+i/t + Pt-L+l,t/tQ Ct (yt — Qff/ t ) 

Ht+\Vt+i + Tt+i Bti't 
O 

o 



o 



(89) 
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The statements in (i)-(iii) easily follow from (89). 
Let Mt be defined by 



Mi 



Pt-Gt [QPt-RtHt 



Pt-i,t/tQ 

Pf-2,t/t Q 

Pt-L,t/fCt 
We also introduce component matrices of Mt as follows 



■ M (0,0) M (0,l) M (0,2) 
M (l,0) M (l,l) M (l,2) 



Pt/tQ'-HtRt 



Pt/tCt T -HtR t y 
Pt-i t/t Ct 



t-i,t/t ' 



- t-2,t/t 



Q' 



Pt-L,t/fCt 



M t 



m| 2 - 0) mI 2 - 1 ' m[ 2 ' 2) 



7,AU0) i,(L,\) , „(L,2) 

M( M) ' M\ ' 



M 



(P,Lh 



M\ 



(U) 



M 



(2,L) 



M\ 



{L,L) 



Concerning Pt+\, we have 



P t +i = A} +1 M,A} +l +T t+l TtQjt T t +i + H t+1 R t+1 H t+1 



A} + M m ^ T *kA m a} + M 0A) 



*Ui M t 



(0,L-1) 






M\ 



(0,0) 



M\ 



(0,1) 



M\ 



(0,1-1) 



M 



(1,0) 



M 



(1,1) 



M 



(U-l) 



M^" 1 ' ^/ Mf- 1 ' ) M t (L - U) 



m; 



(L-1,L-1) 



+ 



Tt+iQtT> + i T +n t+l R t+l H t+l T o o ... o" 



o 
o 



o o ... o 
o o ... o 



o o o ... o 

The final part (iv) can be obtained from the last three equalities. 
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6. Conclusion 

In this chapter, we considered discrete-time linear stochastic systems with unknown inputs 
(or disturbances) and studied three types of smoothing problems for these systems. We 
derived smoothing algorithms which are robust to unknown disturbances from the optimal 
filter for stochastic systems with unknown inputs obtained in our previous papers. These 
smoothing algorithms have similar recursive forms to the standard optimal filters and 
smoothers. Moreover, since our algorithms reduce to those known smoothers derived from 
the Kalman filter when unknown inputs disappear, these algorithms are consistent with the 
known smoothing algorithms for systems without unknown inputs. 

This work was partially supported by the Japan Society for Promotion of Science (JSPS) under 
Grant-in-Aid for Scientific Research (Q-22540158. 
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1. Introduction 

The fast development of network (particularly wireless) technology has encouraged its use 
in control and signal processing applications. Under the control system's perspective, this 
new technology has imposed new challenges concerning how to deal with the effects of 
quantisation, delays and loss of packets, leading to the development of a new networked 
control theory Schenato et al. (2007). The study of state estimators, when measurements are 
subject to random delays and losses, finds applications in both control and signal processing. 
Most estimators are based on the well-known Kalman filter Anderson & Moore (1979). In 
order to cope with network induced effects, the standard Kalman filter paradigm needs to 
undergo certain modifications. 

In the case of missing measurements, the update equation of the Kalman filter depends on 
whether a measurement arrives or not. When a measurement is available, the filter performs 
the standard update equation. On the other hand, if the measurement is missing, it must 
produce open loop estimation, which as pointed out in Sinopoli et al. (2004), can be interpreted 
as the standard update equation when the measurement noise is infinite. If the measurement 
arrival event is modeled as a binary random variable, the estimator's error covariance (EC) 
becomes a random matrix. Studying the statistical properties of the EC is important to 
assess the estimator's performance. Additionally, a clear understanding of how the system's 
parameters and network delivery rates affect the EC, permits a better system design, where 
the trade-off between conflicting interests must be evaluated. 

Studies on how to compute the expected error covariance (EEC) can be dated back at least 
to Faridani (1986), where upper and lower bounds for the EEC were obtained using a constant 
gain on the estimator. In Sinopoli et al. (2004), the same upper bound was derived as the 
limiting value of a recursive equation that computes a weighted average of the next possible 
error covariances. A similar result which allows partial observation losses was presented 
in Liu & Goldsmith (2004). In Dana et al. (2007); Schenato (2008), it is shown that a system in 
which the sensor transmits state estimates instead of raw measurements will provide a better 
error covariance. However, this scheme requires the use of more complex sensors. Most of 
the available research work is concerned with the expected value of the EC, neglecting higher 
order statistics. The problem of finding the complete distribution function of the EC has been 
recently addressed in Shi et al. (2010). 



72 Discrete Time Systems 

This chapter investigates the behavior of the Kalman filter for discrete-time linear systems 
whose output is intermittently sampled. To this end we model the measurement arrival event 
as an independent identically distributed (i.i.d.) binary random variable. We introduce a 
method to obtain lower and upper bounds for the cumulative distribution function (CDF) of 
the EC. These bounds can be made arbitrarily tight, at the expense of increased computational 
complexity. We then use these bounds to derive upper and lower bounds for the EEC. 

2. Problem description 

In this section we give an overview of the Kalman filtering problem in the presence of 
randomly missing measurements. Consider the discrete-time linear system: 

x t+ i = Ax, + w t 
yt = Cx, + v f 

where the state vector x t £ R" has initial condition Xq ~ N(0, Pq), y S W is the measurement, 
w ~ N(0, Q) is the process noise and v ~ N(0,R) is the measurement noise. The goal of the 
Kalman filter is to obtain an estimate Xt of the state Xt, as well as providing an expression for 
the covariance matrix Pj of the error Xt = Xt — Xt. 

We assume that the measurements yt are sent to the Kalman estimator through a network 
subject to random packet losses. The scheme proposed in Schenato (2008) can be used to 
deal with delayed measurements. Hence, without loss of generality, we assume that there is 
no delay in the transmission. Let yt be a binary random variable describing the arrival of a 
measurement at time t . We define that yt = \ when yt was received at the estimator and 7f = 
otherwise. We also assume that yt is independent of 7 S whenever t ^s. The probability to 
receive a measurement is given by 

A = V(y t = 1). (2) 

Let x t i s denote the estimate of Xt considering the available measurements up to time s. Let 
x t \ s = Xt — x t u denote the estimation error and S f i s = E{(x t i s — E{x t i s })(x t i s — E{x t i s })'} 
denote its covariance matrix. If a measurement is received at time t (i.e., if yt = 1)/ the estimate 
and its EC are recursively computed as follows: 

*f|f = *t\t-l+ K t(.yt-Cx t ) (3) 

Z tlt = (I-K t C)X tl t„ 1 (4) 

*f+i|f = A%t\t (5) 

Z t+l]t = A-L t]t A' + Q, (6) 

with the Kalman gain Kt given by 

K t = Z tlt _ 1 C>(CZ tlt _ 1 C> + Q)- 1 . (7) 

On the other hand, if a measurement is not received at time t (i.e., if yt = 0), then (3) and (4) 
are replaced by 



l f|f-l 



(8) 
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S f | ( = £(](_!. (9) 

We will study the statistical properties of the EC E ( | ( _j. To simplify the notation, we define 
Pt = E(i(_j. Then, the update equation of Pt can be written as follows: 

Pf+1 ~l*o(PO, 7t = (10) 

with 

<E> x (P t ) = AP f A' + Q - AP,C'(CP t C' + R) _1 CP t A' (11) 

*o(Pt) = AP t A' + Q. (12) 

We point out that when all the measurements are available, and the Kalman filter reaches its 
steady state, the EC is given by the solution of the following algebraic Riccati equation 

P = APA' + Q- APC'{CPC' + Ry l CPA'. (13) 

Throughout this chapter we use the following notation. For given TsN and < m < 2 — 1, 
the symbol Sj n denotes the binary sequence of length T formed by the binary representation 
of m. We also use S J, (i), i = 1, ■ ■ ■ , T to denote the 2-th entry of the sequence, i.e., 

Sl = {Sl(l), Sj„(2),...,S T m (T)} (14) 

and 

m=^2 k - 1 S T m (k). (15) 

k=i 

(Notice that Si denotes a sequence of length T formed exclusively by zeroes.) We use \S„\ to 
denote the number of ones in the sequence Sj n , i.e., 

\Sl\=£sJ n (k). (16) 

k=1 

For a given sequence S,„, and a matrix P £ R" x,! , we define the map 

4>{P,Sl) =®Sl(T) ®Sl(T-l) ---®SZ(l)( P ) (17) 

where o denotes the composition of functions (i.e. / o g(x) = f(g(x))). Notice that if m is 
chosen so that 

S„ = {7t-\, lt-1, ■ ■ ■ , lt-T), (18) 

then the map ^)(-,S^) updates Pf_r according to the measurement arrivals in the last T 
sampling times, i.e., 

Pt = 4>(Pt-T,S T m ) = * T( .i Vi ° • ..* 7 .-r( p «-r)- (19) 



74 Discrete Time Systems 

3. Bounds for the cumulative distribution function 

In this section we present a method to compute lower and upper bounds for the limit CDF 
F(x) of the trace of the EC, which is defined by 

F{x) = lim F T {x) (20) 

T— >oo 

F T (x) = V(Tr{P T }<x) (21) 

2 T -1 

= £ V ( S ™) H { x - M<P(Po, Si)}) , (22) 

m=0 

where H(-) is the Heaviside step function, and the probability to observe the sequence Sj n is 
given by 

7>(s„ r ,)=Al s >»l(l-A) r -l s >»l. (23) 

The basic idea is to start with either the lowest or the highest possible value of EC, and then 
evaluate the CDF resulting from each starting value after a given time horizon T. Doing so, 

T — T 

for each T, we obtain a lower bound F_ {x) and an upper bound F (x) for F(x), i.e., 

F T (x) < F(x) < F T (x), for all TeR. (24) 

As we show in Section 3.3, both bounds monotonically approach F(x) as T increases. 

To derive these results we make use of the following lemma stating properties of the maps 

4>o(-) and 4>i(-) defined in (11)-(12). 

Lemma 3.1. Let X, Y G K" x " be two positive semi-definite matrices. Then, 

Ol(X) < *o(X). (25) 

IfY > X, 

<5 (V) > *o(X) (26) 

*l(Y) > *i(X). (27) 

Proof: The proof of (25) is direct from (11)-(12). Equation (26) follows straightforwardly since 
<I , o(X) is affine in X. Using the matrix inversion lemma, we have that 

® 1 (X)=A(X- 1 + C'R- 1 C)- 1 A' + Q (28) 

which shows that <t>j (X) is monotonically increasing with respect to X. ■ 



3.1 Upper bounds for the CDF 

The smallest possible value of the EC is obtained when all the measurements are available 
and the Kalman filter reaches its steady state. In this case, the EC P_ is given by (13). Now, 
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fix T, and suppose that m is such that S„, = {/Yt—Ii 7T-2/ • • • > 70 } describes the measurement 
arrival sequence. Then, assuming that Pq > P , from (26)-(27), it follows that Pj > <p(P_, Sj n ). 
Hence, from (22), an upper bound of F(x) is given by 



2'-l 



F (*) = E ? (S^J H (x - Tr{<K£X)} 

m=0 



(29) 



3.2 Lower bounds for the CDF 

A lower bound for the CDF can be obtained using an argument similar to the one we used 
above to derive an upper bound. To do this we need to replace in (22) Tr{<p (Pg, S„)} by an 
upper bound of Tr{ Pj } given the arrival sequence Sj n . To do this we use the following lemma. 

Lemma 3.2. Let m be such that Sj n = {jt-V 7T-2> '" >7o} an d < t\,t2, • • • , tj < T — 1 denote 
the indexes where y^ = 1, i = 1, • • • , I. Define 



O 



CM* 1 " 1 
CA** 



CA l > 



>*Q 



fi-l 
E CAiQA'T-'i+i 

7=0 

E CAiQA' T -'i+l 
7=0 



f/-l 

E C^QA' T - t '+i 
7=0 



and f/ie matrix Ey G RP /x P J , w/zose (i,j)-th submatrix [Ey]; v 6 RP X P z's groen fey 

min{t„f,}-l 



fc=o 



where 



S(i,j) 



0, t^/. 



I/O has full column rank, then 



Pt < P(Si), 



where the S, n -dependant matrix P(S„,) is given by 



~\' T + £ AIQA'I- A T (L v *0)*Z v h' Q + 
7=0 






(30) 



(31) 



(32) 



(33) 



(34) 



-E Q E y ! (E/0)' + A' r - E Q (E,; 1 - E y 1 0(0'E y 1 0)- 1 0'E y 1 ) E' Q , 



with (Ey 2 O) denoting the Moore-Penrose pseudo-inverse o/Ey 2 O Ben-Israel & Greville (2003). 



1 If this assumption does not hold, one can substitute P by Pq without loss of generality. 
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Proof: Let Yj be the vector formed by the available measurements 

Yt= [y' h y' t2 •••<]' 

= Ox + V T , 



where 



V T 



From the model (1), it follows that 



where 



ZpoCA^-)iv j + v h 
Ejro 1 CA'^-Iwj + v t2 



^CA^-hv^ 



v h 



(35) 
(36) 



(37) 



" X T ~ 


~N( 







' 


Ej. E t y 
E^y Ey 



r-i 



E x = A T P A ,r + ^A'QA'I 
;=0 

E xy = A^qO' + E Q 



Ey = OP O' + Ey. 

Since the Kalman estimate Xj at time T is given by, 

f T = E{x T \Y T }, 



(38) 

(39) 

(40) 
(41) 

(42) 



it follows from (Anderson & Moore, 1979, pp. 39) that the estimation error covariance is given 

by 

Substituting (39)-(41) in (43), we have 



(43) 



r-i 



P T = A T P A' T + £ A'QA'! + 
1=0 



- [A T P O' + E Q (OP O' + E y ) [A T P O' + E Q 



A T P - P O' (OP O' + Ey) O'Prj 1 A T + £ A'QA'> + 



r-i 



(44) 



(45) 



H> 



Now, from (19), 



-A T P O' (OP O' + Ey) X E' Q - E Q {OP O' + Ey) l O'P A T 
-EQ^PoO' + Ey)" 1 ^. 



p T = 4>(P ,sl 



(46) 
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Notice that for any Pq we can always find a k such that kl n > Pq, where I n is the identity 
matrix of order n. From the monotonicity of <p(-,S; n ) (Lemma 3.1), it follows that 

P T < lim 4>{kI n ,Sl). (47) 

k— >oo 

We then have that 

P T < P T1 + P T2 + P T3 + P' T3 + P TA , (48) 



with 



P T/1 = lim A T (kl„ - k 2 ikOO' + Ey) 1 O') A ,T (49) 

k— >oo V / 

T-l 

Pt,2 = E A1QA'> 
7=0 

P T3 = - lim kA r O {kOO' + Ey) _1 E' 



P r4 = - lim E (kOO' + Ey) E' . 
Using the matrix inversion lemma, we have that 

P rl = A T lim (V 1 /,, + 0'L- 1 0) A' T (50) 

= A T (b'E^o) A' T . (51) 

It is straightforward to see that Pj- 3 can be written as 

P T3 = - lim A t O' (OO' + Ey*:" 1 ) ~ E' D (52) 

= -A r lim O'E^ (e~*00'E~* +A:" 1 I„ J E~*E' Q . (53) 

From (Ben-Israel & Greville, 2003, pp. 115), it follows that lim^,*, X' (XX' + fc _1 I K ) = X + , for 

_i 
any matrix X. By making X = E„ 5 0, we have that 

P T ,3 = -A T (z v h) Z v h' Q . (54) 



Using the matrix inversion lemma, we have 



-1 



Pta = - I™ E Q E- 1 - E^O O'E^O + JT 1 J„) O'E" 1 E' q (55) 

-1 v— In In'v-lnl r>'v— 1 \ v' 



= E Q (^ - E y '0 [O'Zy'O) O'E^I E' Q 

and the result follows by substituting (51), (54) and (55) in (48). 
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In order to keep the notation consistent with that of Section 3.1, with some abuse of notation 
we introduce the following definition: 

t, a \P(Sl,), if O has full column rank 
4>(°°,Sl)±\ \ m " (56) 

I ooi n , otherwise 

where oo7„ is an n x n diagonal matrix with oo on every entry of the main diagonal. Then, we 
obtain a lower bound for F(x) as follows 

2 T -1 

£ T « = E V ( S <«) H { x - Tr{<M~X)}) . (57) 

m=0 

3.3 Monotonic approximation of the bounds to F(x) 

In this section we show that the bounds F_ (x) and F (x) in (24) approach monotonically F(x), 
as T tends to infinity. This is stated in the following theorem. 

Theorem 3.1. We have that 

L T+1 (x) > F T {x) (58) 

F T+ \x) < F T (x). (59) 

Moreover, the bounds F_ (x) and F (x) approach monotonically the true CDF F(x) as T tends to oo. 

Proof: Let Sf n be a sequence of length T. From (17) and Lemma 3.1 and for any Pq > 0, we 
have 

4>(Po,{sl,o}) = <K*o(Po),s£) < 4>(«>,sl). (60) 

From the monotonicity of <p{-, Sj n ) and <t>o(')/ stated in Lemma 3.1 we have 

<P(PoASlO}) = i>(^ Q (P ),Sl) > cp(P ,S T m ) (61) 

which implies that 

<p(°°,{SlO})><p(oo,Sl). (62) 

From (60) and (62), we have 

<M^{sLo}) = <Koo,s„ r ,). (63) 

Also, if the matrix O (defined in Lemma 3.2) resulting from the sequence Sj n has full column 
rank, then so has the same matrix resulting from the sequence {Sf n , 1}. This implies that 

0(oo,{sLl}) <<*>(<*>, S£)- (64) 

Now, from Lemma 3.1, <5o(£) ^ Ej an( i therefore, 

<P(L{Sl,0})=cp(® (P),Sj n ) (65) 

><p(P,Sl). (66) 
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Fig. 1. Upper and lower bounds for the Error Covariance. 
Also, since <&i (P) = P, we have that 

•p{P,{Sl,\}) = <p{<p 1 {P),S T m ) 

Hence, for any binary variable 7, we have that 

<PK{s„ T „7}) <4>(*>,sl) 

<P(E J {S T m ,y})>'P(P,Sl). 



(67) 
(68) 



(69) 
(70) 



Now notice that the bounds (29) and (57) only differ in the position of the step functions H( 
Hence, the result follows from (69) and (70). 



3.4 Example 

Consider the system below, which is taken from Sinopoli et al. (2004), 



Q 



1.25 
1 1.1 

20 
20 



C 



(71) 



R = 2.5, 



with A = 0.5. In Figure 1 we show the upper bound F (x) and the lower bound F_ (x), for 
T = 3, T = 5 and T = 8. We also show an estimate of the true CDF F(x) obtained from a 
Monte Carlo simulation using 10, 000 runs. Notice that, as T increases, the bounds become 
tighter, and for T = 8, it is hard to distinguish between the lower and the upper bounds. 
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4. Bounds for the expected error covariance 

In this section we derive upper and lower bounds for the trace G of the asymptotic EEC, i.e., 

G= limTr{E{P f }}. (72) 

f^oo 

Since Pt is positive-semidefinite, we have that, 

/■CO 

Tr{E{P,}}= / (1-F t (x))dx. (73) 

JO 

Hence, 2 

/•CO 

G = / (1- lim F f (%))<*»: (75) 

JO M-oo 

/•CO 

= / (1-F(x))dx (76) 

4.1 Lower bounds for the EEC 

In view of (76), a lower bound for G, can be obtained from an upper bound of F(x). One 

such bound is F (x), derived in Section 3.1. A limitation of F (x) is that F (x) = 1, for all 

* > <P(E/ Sq), hence it is too conservative for large values of x. To go around this, we introduce 

an alternative upper bound for F(x), denoted by F*(x). 

Our strategy for doing so is to group the sequences Sj n , m = 0, 1, • • • ,2 — 1, according to the 

number of consecutive lost measurements at its end. Then, from each group, we only consider 

the worst sequence, i.e., the one producing the smallest EEC trace. 

Notice that the sequences Sj, with m < 2 T ~ Z , < z < T, are those having the last z elements 

equal to zero. Then, from (25) and (26), it follows that 

argminTr{<p(X,s£)} = 2 T ~ Z - 1, (77) 

0<m<2 T -= 

i.e., from all sequences with z zeroes at its end, the one that produces the smallest EEC trace 
has its first T — z elements equal to one. Using this, an upper bound for F(x) is given by 

F(x)<F,(x)il-(l-A) k W (78) 

where 

JO, x<P 

jminjj:Tr (<£(£, S ; )) > x} , x > P. 



2 



Following the argument in Theorem 3.1, it can be verified that (1 — F'(x)) < F(x) with 

[F(x) x > Tr{P }. 

Hence, using Lebesgue's dominated convergence theorem, the limit can be exchanged with the integral 
whenever / (1 — F(x))dx < oo, i.e., whenever the asymptotic EEC is finite. 
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— T — T 

We can now use both F (x) and F* (x) to obtain a lower bound G for G as follows 

G T = l-min{F(x),FJx)}dx. (80) 

JO 

The next lemma states the regions in which each bound is less conservative. 

Lemma 4.1. The following properties hold true: 

F T {x) < F*(x), Vx < Tr (<p{P, S T )) (81) 

F T (x) > F*(x), Vx > Tr (<f(P,S T )) . (82) 

Proof: Define 

Z(i,j)±-hU(P,sh). (83) 



J 1 

To prove (81), notice that F (x) can be written as 



F T (x)= 2 £ P(S[). (84) 

;'=o 

j:Z(j,T)<x 



Substituting x = Z(0, K) we have for all 1 < K < T 

F i (Z(0,_K))= ^ P( S J) (85) 

/:Z(7,T) <Z(0,K) 

= 1- £ P(Sj) (86) 

;=0 

;:Z(;,r)>Z(0,K) 

Now, notice that the summation in (86) includes, but is not limited to, all the sequences 
finishing with K zeroes. Hence 

2 T -1 

£ V(SJ)>(1-\) K (87) 

7=0 

j:Z(j,T)>Z(0,K) 

and we have 

F T (Z(0,K)) <1-(1-A) K (88) 

= F*(Z(0,J0). (89) 

Y 

Proving (82) is trivial, since F (x) = l,x > Z(0, T). ■ 

We can now present a sequence of lower bounds G , T G N, for the EEC G. We do so in the 

next theorem. 
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Theorem 4.1. Let Ey, < j < 2 T denote the set of numbers Tr ($(P, Sj,)), < m < 2 T , arranged 
in ascending order, (i.e., £, = Tr (</>(£, Sj,.) ),/or some nij, and E\ < E\ < • • • < £ 2 t). For eac?! 
< j < 2 , let tt; = Ej-lg ^(S^ ). Also define Eg = tzq = 0. Then, G defined in (80) is given by 

G T = G[ + Gj (90) 

where 

Gj= £(l-7r ; )(£ /+1 -£ ; ) (91) 

7=0 

G J = £(1 - Ay'Tr I A' (APA' + Q - P) A'' } (92) 

/=r 

Moreover, if the following condition holds 

max|eig(A)| 2 (l-A) < 1, (93) 

and A is diagonalizable, i.e., it can be written as 

A = VDV- 1 , (94) 

with D diagonal, then, 

T—l 
Q_l = Tr{r} - £ (1 - Ay'Tr { A' (APA' + Q - P) A 'A (95) 

7=0 

where 

r 4 fx^V'- 1 ®7Ja (x 1/2 v'- 1 ® v) ' (96) 

X = APA 1 + Q-P. (97) 

Also, the n 2 x « 2 matrix A is smcm fnaf its i,j-th entry [A],- , z's given by 

[A] ■ 4 1 _ ^ , (98) 

U l-(l-A)[C?],[cT], 

ry/zere D denotes a column vector formed by stacking the columns ofD, i.e., 

I?^[D] U ...[D]„ ;1 [D]„...[D] M ]' (99) 

Proof: In view of lemma 4.1, (90) can be written as 

rZ{0,T) _ T f-co _ 

G T = (1-F (x))dx+ <l-FJx))dx (100) 

JO JZ{0,T) 
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— T 
Now, F (x) can be written as 

F (*) = n i(xy K x ) = max {* : E i < *}• 
In view of (101), it is easy to verify that 

f Z(0,T) _ T 2^ 

/ 1 -F (x)dx = £(1 - n^Ej - Ey_i) = G 

JO j =l 

The second term of (90) can be written using the definition of F+(x) as 

/•CO CO 

J 1 - F(x)dx = E (1 - A)' (Z(0,; + 1) - Z(0,;)) 



i=T 



00 r i 

£ (1 - A)^Tr J A^ (APA' + Q-P)A'lj 



Gl 



(101) 
(102) 

(103) 

(104) 
(105) 



and (90) follows from (100), (102) and (105). 

To show (95), we use Lemma 7.1 (in the Appendix), with b = (1 - A) and X = APA' + Q - P, 

to obtain 

£(1 - A)'Tr | A' (APA' + Q - P) A'U = Tr{r}. (106) 

/=0 

The result then follows immediately. ■ 



4.2 Upper bounds for the EEC 

Using an argument similar to the one in the previous section, we will use lower bounds of the 

— T,N 
CDF to derive a family of upper bounds G ' ,T<N6 N, of G. Notice that, in general, there 

exists 5 > such that 1 — £ (x) > 5, for all x. Hence, using F_ (x) in (76) will result in G being 

infinite valued. To avoid this, we will present two alternative lower bounds for F(x), which 

we denote by F^ N (x) and F$f(x). 

Recall that A £ E" x ", and define 





' 


( 


C 
CA 


\ 


\ 


min < 


k : rank 




CA 1 




= n 






\ 


CA k -\ 


) 





N 



The lower bounds £*' (x) and F_o ( x ) are stated in the following two lemmas. 
Lemma 4.2. Let T < N G N, a;ifft N < T and N satisfying 

|S^|>N ^Tr{^(oo,S^)}<oo. 



(107) 



(108) 
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For each T < n < N, let 

P*{n)± max <p(oo,S'^) (109) 

m:\S'i,\ = N 

p*(n)^Tr(P*(w)). (110) 

Then, for all p*(T) <x< p*{N), 

F(x) > Fl' N (x), (111) 

where, for each T < n < N and allp*(n) < x < p* (n + 1), 

No-l .,, 

£l' N W = l- E A'(l-A)-' 7j ^- 7yi . (112) 

Remark 4.1. Lemma 4.2 above requires the existence of an integer constant N satisfying (108). Notice 
that such constant always exists since (108) is trivially satisfied by Nq. 

Proof: We first show that, for all T < n < N, 

p*(n)<p*(n + l). (113) 

To see this, suppose we add a zero at the end of the sequence used to generate p*(n). Doing 
so we have 

P*(n) <<J> (p*(h)) <P*(n + l). (114) 

Now, for a given n, we can obtain a lower bound for F"(x) by considering in (57) that 
Tr(4>(oo,S£,)) = oo, whenever \S',' n \ < N . Also, from (25) we have that if |S*| > N , 
then Tr(<p(co, SJJ,)) < p*(n). Hence, a lower bound for F(x) is given by 7^(15^1 < No), for 
x > f*{n). 

Finally, the result follows by noting that the probability to observe sequences S" n with m such 
that | S", | < N is given by 

No-l „i 

V(\S n m \ < N ) = 1 - £ A'(l - A)"-'——, (115) 

; =0 ( -V" l >- 

since A (1 — A)" - is the probability to receive a given sequence S", with \S" n \ = I, and the 
number of sequences of length n with / ones is given by the binomial coefficient 

(116) 



IJ l\{n-l)V 



-* _ N °- 1 fN\ 

Lemma 4.3. Let N, P (N) and p" (N) be as defined in Lemma 4.2, and let L = J] I . Then, 

«=0 \ n J 

for all x > p*{N), 

F(x) > £*(*), (117) 



On the Error Covariance Distribution for Kalman Filters with Packet Dropouts 85 

where, for each n e N and all <p(F* (N), Sg" 1 ) <x< <p(F*(N),S%), 

Fj*(x) = 1 - u'M"z (118) 

with the vectors n,z 6 R L defined by 

u = [1 1 ... l]' (119) 

z = [l ■■■ 0]'. (120) 

The i,j-th entry of the matrix M 6 R LxL is given by 



[M]i,j 



A, Zf=U+{Zf,l) 



f = U+( 7 
0, otherwise 



1-A, Z™ = !I+(Z™,0) (121) 



where Z„ jr m = 0, • • • ,L — 1 denotes the set of sequences of length N with less than Nq ones, with 
Zg" = Sq, fowf otherwise arranged in any arbitrary order (i.e., 

\ z m\ < Noforallm = 0,- ■ ■ ,L - 1. (122) 



and Z^ = S„ ,for some n m G {0, ■ ■ ■ ,2 N — 1}). Also, for 7 e {0,1}, the operation li + (Zj,,7) is 
defined by 

U+(Zl, 7 ) = {Zj„(2),Z T m (3),---,Zl(T), 7 }. (123) 

Proof: The proof follows an argument similar to the one used in the proof of Lemma 4.2. 
In this case, for each n, we obtain a lower bound for F"(x) by considering in (57) that 
Tr(<f(oo, SJJ,)) = 00, whenever S", does not contain a subsequence of length N with at least 
Nq ones. Also, if S" n contains such a subsequence, the resulting EC is smaller that or equal to 

*(»,{S&,S&}) = M(«,S& ),SrJ) (124) 

= tf>(P*(N),Sg), (125) 

where S^f, denotes the sequence required to obtain P (N). 

To conclude the proof we need to compute the probability pM,n or receiving a sequence of 
length N + n that does not contain a subsequence of length N with at least Nq ones. This is 
done in Lemma 7.2 (in the Appendix), where it is shown that 

p N ,„ = u'M n z. (126) 

■ 

— T N 
Now, for a given T and N, we can obtain an upper bound G ' for G using the lower bounds 

E T (x), EI- N (x) and F^{x), as follows 

G T,N = /' 00 l-max{£ T (x),£j' N (x),£^(x)}dx. (127) 
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We do so in the next theorem. 

Theorem 4.2. Let T and N be two given positive integers with Nq < T < N and such that for all 
< m < 2 N , \S„\ > Ng => <f (oo,S^) < oo. Let } be the number of sequences such that O(SJ^) has 
full column rank. Let Eg = and £,-, < ;' < / denote the set of numbers Tr (<f (°o, S^)), < m < }, 

arranged in ascending order, (i.e., Ej = Tr (tp(co,Sm) J, for some mj, and Eq < E\ < • • • < Et). For 

each < j < J, let n; = ^LnViSj), and let M, u and v be as defined as in Lemma 4.3. Then, an 
upper bound for the EEC is given by 

G < G T,N , (128) 



— T N , — T — T N — N , 

G ' =Tr(Gj +G2' +G3), 


(129) 


and 




T ] 

G{ =£(l-7ry)(E^ + i-Ey) 

;=0 


(130) 




(131) 


CO 

G3 = ]Tu'M N+ iz{Ai(AP*(N)A' + Q-P*(N))A'i}. 


(132) 


Moreover, if A is diagonalizable, i.e. 

A = VDV- 1 , 


(133) 


with D diagonal, and 

max eig(A) \ 2 p < 1, 


(134) 


where 

p = (max |svM|), 


(135) 


then the EEC is finite and 

G3 < u'M N zTr(T*), 


(136) 


where 




t* 4 (x 1/2 V- 1 ® v) a (x 1/2 y'- a ® v)' 


(137) 


X = APA' + Q-P. 


(138) 



Also, the i,j-th entry [A],- 1 of the n 2 x n 2 matrix A is given by 



[A]y 



V2 N o-l 

1-P0]i0)i' 



(139) 



Proof: First, notice that Fj(x) is defined for all x > 0, whereas E*( x ) is defined on the range 
P*{T) <x< P*(N) and Fj(x) onF(N) < x. Now, for all x > p*{T), we have 
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N -l t| 

F T (x) = £ r(SJ)=l- £ A'(1-A) T -'-— — -, (140) 

;:|S/|>N '=o '^ J l> - 

which equals the probability of receiving a sequence of length T with Nq or more ones. Now, 
for each integer 1 < n < N — T, and for ~p*(T + n) < x <p*(T + n + 1), Fj' (x) represents 
the probability of receiving a sequence of length T + n with more than or exactly Nq ones. 
Hence, Fj' (x) is greater than £ T (x) on the range P (T) < x < P (N). Also, F^ (x) measures 
the probability of receiving a sequence of length N with a subsequence of length T with Nq or 
more ones. Hence, it is greater than F_ (x) on P (N) < x. Therefore, we have that 

(F T (x), x<rm 

max{F T (x),Fl N (x),F^(x)} = I fJ< n (x), p*(T) < x < p*(N) (141) 

[f2(x), p*(N)<x. 

We will use each of these three bounds to compute each term in (129). To obtain (130), notice 
that F T (x) can be written as 

F T (x) = n i(x) , i(x) = max{i : E, < x}. (142) 

In view of the above, we have that 

f [T \l-l T (x))dx = £(1 - 7ij)(E j+1 - Ej) = G[. (143) 

Using the definition of £*' (x) in (112) we obtain 

rp'IN) N-lNo-1 -| 

/ (l-El' N (x))dx= £ £ A'(l - \y-l—±— (P*(/+l) -?*(/)) (144) 

= G 2 T ' N . (145) 

Similarly, the definition of Eo{ x ) m (118) can be used to obtain 

/ (l-£^(x))dx= rM'M^zTr{A^(AP*(N)A' + Q-P*(N))A^} = G3 . (146) 

Jp"(N) j =Q 

To conclude the proof, notice that 

uM j z = <u,M'z> (147) 

< ||m|| 2 ||M'z||2 (148) 

< ||«||2l|A^'||||z|| 2 (149) 

< ||m|| 2 ||M|H|z||2 (150) 
= ||w|| 2 (maxsvMy||z||2 (151) 



\/2 N ° -l(maxsvMy. (152) 
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0.7 



0.5 
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A = 0.5 



T = 8 

- Shi et al. 



10 



15 



20 



Fig. 2. Comparison of the bounds of the Cumulative Distribution Function. 

where max svM denotes the maximum singular value of M. Then, to obtain (136), we use the 
result in Lemma 7.1 (in the Appendix) with b = maxsvMandX = AP* (N) A' + Q — P* (N) . ■ 



5. Examples 

In this section we present a numerical comparison of our results with those available in the 
literature. 



5.1 Bounds on the CDF 

In Shi et al. (2010), the bounds of the CDF are given in terms of the probability to observe 
missing measurements in a row. Consider the scalar system below, taken from Shi et al. (2010). 



1.4, C = 1, Q = 0.2, R = 0.5 



(153) 



We consider two different measurement arrival probabilities (i.e., A = 0.5 and A = 0.8) and 
compute the upper and lower bounds for the CDF. We do so using the expressions derived 
in Section 3, as well as those given in Shi et al. (2010). We see in Figure 2 how our proposed 
bounds are significantly tighter. 

5.2 Bounds on the EEC 

In this section we compare our proposed EEC bounds with those in Sinopoli et al. (2004) 
and Rohr et al. (2010). 
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Bound 


Lower 


Upper 


From Sinopoli et al. (2004) 


4.57 


11.96 


From Rohr et al. (2010) 


- 


10.53 


Proposed 


10.53 


11.14 



Table 1. Comparison of EEC bounds using a scalar system. 



Bound 


Lower 


Upper 


From Sinopoli et al. (2004) 


2.15 x 10 4 


2.53 x 10 b 


From Rohr et al. (2010) 


- 


1.5 x 10 5 


Proposed 


9.54 x 10 4 


3.73 x 10 b 



Table 2. Comparison of EEC bounds using a system with a single unstable eigenvalue. 

5.2.1 Scalar example 

Consider the scalar system (153) with A = 0.5. For the lower bound (90) we use T = 14, and 
for the upper bound (129) we use T = N = 14. Notice that in the scalar case No = 1, that is, 
whenever a measurement is received, an upper bound for the EC is promptly available and 
using N > T will not give any advantage. Also, for the upper bound in Rohr et al. (2010), 
we use a 'window length of 14 sampling times (notice that no lower bound for the EEC is 
proposed in Rohr et al. (2010)). 

In Table 1 we compare the bounds resulting from the three works. We see that although the 
three upper bounds are roughly similar, our proposed lower bound is significantly tighter 
than that resulting from Sinopoli et al. (2004). 

5.2.2 Example with single unstable eigenvalue 

Consider the following system, taken from Sinopoli et al. (2004), where A = 0.5 and 



"1.25 1 
0.9 7 
0.6 

R = 2.5 



C'= [1 2] 
Q = 201. 



(154) 



Table 2 compares the same bounds described above, with T = 10 and N = 40. The same 
conclusion applies. 



6. Conclusion 

We considered a Kalman filter for a discrete-time linear system whose output is intermittently 
sampled according to an independent sequence of binary random variables. We derived 
lower and upper bounds for the CDF of the EC, as well as for the EEC. These bounds can be 
made arbitrarily tight, at the expense of increased computational complexity. We presented 
numerical examples demonstrating that the proposed bounds are tighter than those derived 
using other available methods. 
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7. Appendix 

Lemma 7.1. Let < b < 1 be a scalar, X G E" x " be a positive-semidefinite matrix and A 6 R" x ™ 
foe diagonalizable, i.e., it can be written as 



with D diagonal. If 
then, 

where 



A = VDV-\ 
max eig(A) 2 b < 1, 



M=Tr( 



Tr £ b'A'XA'' = Tr(r) 



(155) 
(156) 

(157) 
(158) 



roz'ffe g) denoting the Kronecker product. The n 2 x n 2 matrix A is such that its i,j-th entry [A], ,• is 
given by 

[A] ■ 4 A _, , (159) 

' l-b[D],[D] ; 

tufeere D denotes a column vector formed by stacking the columns ofD, i.e., 
^=[[D] 1 , 1 ---[D]„, 1 [D] 1 , 2 ...[D]„ / „]'. 

Proof: For any matrix 

B l,l Bl,2 • ■ • B l,n " 
B 2,l #2,2 • ■ ■ 62,11 

S n ,l B H,2 • ■ • B«,H . 

with By 6 R" XH , we define the following linear transformation 



(160) 



(161) 



2>„(B) = £B W . 



(162) 



Now, substituting (155) in (157), and using the vectorization operation ■ defined above we 
have 



J^biA>XA'i = ^biVDiv^X 1 ' 2 [VDiV^X 1 ' 2 
;=0 ;=0 






b'VDiv^X^ 2 (vD'V- l X l/2 



V n 



X l/2 V'- 1 9 V) £ ^d/' X 172 !/'- 1 ® V 

7=0 



(163) 
(164) 

(165) 
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where the last equality follows from the property 

Ab£= (C'<8 A)t. (166) 

Let 6ij denote the i,/-th entry of buu', and pow (Y,j) denote the matrix obtained after 

elevating each entry of Y to the /-th power. Then, if every entry of buu' has magnitude 
smaller than one, we have that 



00 


J. > ' 




L y 


WW 


= 


i=o 




hi 



— >— > 
^pow(fcDD',7) 

;' =0 

1 



V 



1 - hi 



(167) 
(168) 



where [Y]; y denotes the i,/-th entry of Y. Notice that D D ' if formed by the products of the 
eigenvalues of A, so the series will converge if and only if 



max eig(A) b < 1. 



Putting (168) into (165), we have that 

00 

£ biA'XA'' = v„ 

7=0 

= V n (T) 

and the result follows since Tr{X>„{Y}} = Tr{Y}. 



X 1/2 V'- 1 ®v)A (x l/2 V'- 1 8 V 



(169) 

(170) 
(171) 



Lemma 7.2. Let u, z, Nq, L and M be as defined in Lemma 4.3. The probability pjv /H of receiving 
a sequence of length N + n that does not contain a subsequence of length N with at least No ones is 
given by 



PN,n = UM 



(172) 



Proof: 

Let Z„, m = 0, ■ ■ ■ , L — 1, and U+ [Zj n , 7) be as defined in Lemma (4.3). Also, for each N, t 6 
N, define the random sequence Vj" = {jtr'Yt-lr • • /7f-N+l}- Let Wf be the probability 
distribution of the sequences Z^, i.e. 



W t 



7>(V t N 
p(yN 

V(V t N = 



Zo N ) 
Zf) 



Z N ) 



(173) 



One can write a recursive equation for Wf + i as 



W f+ i = MW|. 



(174) 
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Hence, for a given n, the distribution W n of V„ is given by 

W„ = M"W . (175) 

To obtain the initial distribution Wg, we make V_ N = Zq, which gives 

W_ N = z. (176) 

Then, applying (175), we obtain 

W = M N z. (177) 

Finally, to obtain the probability pj\r,n/ we add all the entries of the vector W n by 
pre-multiplying W„ by u. Doing so, and substituting (177) in (175), we obtain 

p Nrll = uM N+n z. (178) 
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1. Introduction 

State estimation plays an important role in any application dealing with modeling of dynamic 
systems. In fact, many fields of knowledge use a mathematical representation of a behavior 
of interest, such as, but not limited to, engineering (mechanical, electrical, aerospace, civil and 
chemical), physics, economics, mathematics and biology Simon (2006). 

A typical system dynamics can be represented as a transfer function or using the space-state 
approach. The state-space approach is based on the time-evolution of the "states" of the 
system, which are considered all the necessary information to represent its dynamic at the 
desired point of operation. That is why the knowledge about the states of a model is so 
important. However, in real applications there can be two reasons where the states of a system 
can not be measured: a) measuring a state implies in the need of a sensor. In order to measure 
all the states of a system it will be required a large amount of sensors, making the project 
more expensive and sometimes unfeasible. Usually the whole cost includes not only the price 
of the sensors, but also modifies the project itself to fix all of them (engineering hours, more 
material to buy, a heavier product), b) Some states are impossible to be physically measured 
because they are a mathematically useful representation of the system, such as, the attitude 
parameterization of an aircraft altitude. 

Suppose we have access to all the states of a system. What can we do with them? As the states 
contain all the information necessary about the system, one can use them to: 

a) Implement a state-feedback controller Simon (2006). Almost in the same time the state 
estimation theory was being developed, the optimal control was growing in popularity 
mainly because its theory can guarantees closed loop stability margins. However, the 
Linear-Quadratic-Gaussian (LQG) control problem (the most fundamental optimal control 
problem) requires the knowledge of the states of the model, which motivated the development 
of the state estimation for those states that could not be measured in the plant to be controlled. 

b) Process monitoring. In this case, the knowledge of the state allows the monitoring of the 
system. This is very useful for navigation systems where it is necessary to know the position 
and the velocity of a vehicle, for instance, an aircraft or a submarine. In a radar system, this 
is its very purpose: keep tracking the position and velocity of all targets of interest in a given 
area. For an autonomous robot is very important to know its current position in relation to an 
inertial reference in order to keep it moving to its destiny. For a doctor is important to monitor 
the concentration of a given medicine in his patient. 
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c) Process optimization. Once it is possible to monitor the system, the natural consequence 
is to make it work better. An actual application is the next generation of smart planes. 
Based on the current position and velocity of a set of aircraft, it is possible to a computer 
to better schedule arrivals, departures and routes in order to minimize the flight time, which 
also considers the waiting time for a slot in an airport to land the aircraft. Reducing the 
flight time means less fuel consumed, reducing the operation costs for the company and the 
environmental cost for the planet. Another application is based on the knowledge of the 
position and velocities of cell phones in a network, allowing an improved handover process 
(the process of transferring an ongoing call or data session from one channel connected to 
the core network to another), implying in a better connection for the user and smart network 
resource utilization. 

d) Fault detection and prognostics. This is another immediate consequence of process 
monitoring. For example, suppose we are monitoring the current of an electrical actuator. 
In the case this current drops below a certain threshold we can conclude that the actuator 
is not working properly anymore. We have just detected a failure and a warning message 
can be sent automatically. In military application, this is essentially important when a system 
can be damaged by exterior reasons. Based on the knowledge of a failure occurrence, it is 
possible to switch the controller in order to try to overcome the failures. For instance, some 
aircraft prototypes were still able to fly and land after losing 60% of its wing. Thinking about 
the actuator system, but in a prognostics approach, we can monitor its current and note that 
it is dropping along the time. Usually, this is not an abrupt process: it takes so time to the 
current drop below its acceptable threshold. Based on the decreasing rate of the current, one 
is able to estimate when the actuator will stop working, and then replace it before it fails. 
This information is very important when we think about the safety of a system, preventing 
accidents in cars, aircrafts and other critical systems. 

e) Reduce noise effect. Even in cases where the states are measured directly, state estimation 
schemes can be useful to reduce noise effect Anderson & Moore (1979). For example, 
a telecommunication engineer wants to know the frequency and the amplitude of a sine 
wave received at his antenna. The environment and the hardware used may introduce 
some perturbations that disturb the sin wave, making the required measures imprecise. A 
state-state model of a sine wave and the estimation of its state can improve precision of the 
amplitude and frequency estimations. 

When the states are not directly available, the above applications can still be performed 
by using estimates of the states. The most famous algorithm for state estimation is the 
Kalman filter Kalman (1960). It was initially developed in the 1960s and achieved a wide 
success to aerospace applications. Due its generic formulation, the same estimation theory 
could be applied to other practical fields, such as meteorology and economics, achieving 
the same success as in the aerospace industry. At our present time, the Kalman filter is the 
most popular algorithm to estimate the states of a system. Although its great success, there 
are some situations where the Kalman filter does not achieve good performance Ghaoui & 
Clafiore (2001). The advances of technology lead to smaller and more sensible components. 
The degradation of these component became more often and remarkable. Also, the number 
and complexity of these components kept growing in the systems, making more and more 
difficult to model them all. Even if possible, it became unfeasible to simulate the system with 
these amounts of details. For these reasons (lack of dynamics modeling and more remarkable 
parameters changes), it became hard to provide the accurate models assumed by the Kalman. 
Also, in a lot of applications, it is not easy to obtain the required statistic information about 
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noises and perturbations affecting the system. A new theory capable to deal with plant 
uncertainties was required, leading robust extensions of the Kalman filter. This new theory is 
referred as robust estimationGhaoui & Clafiore (2001). 

This chapter presents a robust prediction algorithm used to perform the state estimation of 
discrete time systems. The first part of the chapter describes how to model an uncertain 
system. In the following, the chapter presents the new robust technique used when dealing 
with linear inaccurate models. A numerical example is given to illustrate the advantages of 
using a robust estimator when dealing with an uncertain system. 

2. State Estimation 

The Estimation Theory was developed to solve the following problem: given the values of 
a observed signal though time, 1 also known as measured signal, we require to estimate 
(smooth, correct or predict) the values of another signal that cannot be accessed directly or 
it is corrupted by noise or external perturbation. 

The first step is to establish a relationship (or a model) between the measured and the 
estimated signal. Then we shall to define the criteria we will use to evaluate the model. In this 
sense, it is important to choose a criteria that is compatible with the model. The estimation is 
shown briefly at Figure 1. 




Estimator 
Plant 



X 



Fig. 1. Block diagram representing the estimation problem. 

At Figure 1, we wish to estimate signal x. The signal y are the measured values from the plant. 
The signal w indicate an unknown input signal and it is usually represented by an stochastic 
behavior with known statistical properties. The estimation problem is about designing an 
algorithm that is able to provide x, using the measures y, that are close of X for several 
realizations of y. This same problem can also be classically formulated as a minimization 
of the estimation error variance. At the figure, the error is represented by e and can be defined 
as x minus X. When we are dealing with a robust approach, our concern is to minimize an 
upper for the error variance as will be explained later on this chapter. 

The following notation will be used along this chapter: R" represents the M-dimensional 
Euclidean space, W xm is the set of real n x m matrices, E {•} denotes the expectation operator, 
cov {•} stands for the covariance, Z represents the pseudo-inverse of the matrix Z, diag {•} 
stands for a block-diagonal matrix. 

3. Uncertain system modeling 

The following discrete-time model is a representation of a linear uncertain plant: 

*k+l = A Aik x k + w k , (1) 

Vk = C hik x k + v k , (2) 



1 Signal here is used to define a data vector or a data set. 
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where x k € R" v is the state vector, y k G R"" stands for the output vector and w k G R"* 
and v k 6 R"" are the output and measurement noises respectively. The uncertainties are 
characterized as: 



1. 



Additive uncertainties at the dynamic represented as A^ k = A k + A A/ c , where A k is the 
known, or expected, dynamic matrix and AA k is the associated uncertainty. 

Additive uncertainties at the output equation represented as C^ k = C k + AQt, where C k is 
the known output matrix and AQ. characterizes its uncertainty. 

Uncertainties at the mean, covariance and cross-covariance of the noises w k and v k . We 
assume that the initial conditions {xg} and the noises {w k , v k } are uncorrelated with the 
statistical properties 
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(4) 



where W k , V k and Xq denotes the noises and initial state covariance matrices, S k is the cross 

covariance and S k : is the Kronecker delta function. 

Although the exact values of the means and of the covariances are unknown, it is assumed 

that they are within a known set. The notation at (5) will be used to represent the covariances 

sets. 



w k ew k , v k ev k , s k es k 



(5) 



In the next sub section, it will be presented how to characterize a system with uncertain 
covariance as a system with known covariance, but with uncertain parameters. 



3.1 The noises means and covariances spaces 

In this sub section, we will analyze some features of the noises uncertainties. The approach 
shown above considered correlated w k and v k with unknown mean, covariance and cross 
covariance, but within a known set. As will be shown later on, these properties can be 
achieved when we define the following noises structures: 



w k '■= B Aiv,k w k + B Av,k v k, 
®k '■= D Aw,k w k + D Av,k v k- 



(6) 
(7) 



Also here we assume that the initial conditions {xq} and the noises {w k } , {v k } are 
uncorrelated with the statistical properties 
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where W k , V k and Xq denotes the noises and initial state covariance matrices and S k stands for 
the cross covariance matrix of the noises. 

Therefore using the properties (8) and (9) and the noises definitions (6) and (7), we can note 
that the noises w k and v k have uncertain mean given by 



£ {w k } = B AW)k w k + B AvJ( v k , 
E {v k } = D AWrk w k + D AV/k v k . 



(10) 
(11) 



Their covariances are also uncertain and given by 
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(12) 



Using the descriptions (6) and (7) for the noises, we obtain 



S k hj v khj 
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D Aw k D Avk 
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(13) 



The notation at (13) is able to represent noises with the desired properties of uncertain 
covariance and cross covariance. However we can consider some simplifications and achieve 
the same properties. There are two possible ways to simplify equation (13): 

1. Set 



BAw,k B Avk 
E>Aw,k E>Av,k 



?Aw,k 
D 





Av,k 



(14) 



In this case, the covariance matrices can be represented as 
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(15) 



2. The other approach is to consider 
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(16) 



In this case, the covariance matrices are given by 



W k 5 k j Sj5 k j 
S[4 ; V k 6 kj 
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(17) 



So far we did not make any assumption about the structure of noises uncertainties at (6) 
and (7). As we did for the dynamic and the output matrices, it will be assumed additive 
uncertainties for the structure of the noises such as 
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(18) 
(19) 
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where B 



w,k/ "v,k 



D 



vk 



and D vk denote the nominal matrices. Their uncertainties are 
represented by AB mk , AB vk , AD wk and AD vk respectively. Using the structures (18)-(19) for 
the uncertainties, then we are able to obtain the following representation 



%>k = ( B iv,k + bBwjc) w k + (B v ,k + bB v ,k) °k> 
h = (Pwjc + AD wk) Wk + (Dvk + AD v,k) v k - 



(20) 
(21) 



In this case, we can note that the mean of the noises depend on the uncertain parameters of 
the model. The same applies to the covariance matrix. 

4. Linear robust estimation 

4.1 Describing the model 

Consider the following class of uncertain systems presented at (l)-(2): 



*;t+i = (A k + AA k ) x k + w k , 
Vk = (Qc + AQ) x k + v k , 



(22) 
(23) 



where x k G R"* is the state vector, y k £ R"v is the output vector and w k g W x and v k G 
R"" are noise signals. It is assumed that the noise signals w k and v k are correlated and their 
time-variant mean, covariance and cross-covariance are uncertain but within known bounded 
sets. We assume that these known sets are described as presented previously at (20)-(21) with 
the same statistical properties as (8)-(9). 
Using the noise modeling (20) and (21), the system (22)-(23) can be written as 



x k +i = (A k + AA k ) x k + (B l0rk + hB wM ) w k + (B Prk + AB Vik ) v k , 
Vk = (Qc + AQ) x k + [p Wik + AD Wrk ) w k + (D Vik + AD Vik ) v k . 

The dimensions are shown at Table (1). 
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Table 1. Matrices and vectors dimensions. 

The model (24)-(25) with direct feed through is equivalent to one with only one noise vector at 
the state and output equations and that w k and v k could have cross-covariance Anderson & 
Moore (1979). However, we have preferred to use the redundant noise representation (20)-(21) 
with w k and v k uncorrelated in order to get a more accurate upper bound for the predictor 
covariance error. The nominal matrices A k , B wk , B vk , C k , D wk and D v k are known and the 
matrices AA k , AB W k , AB V k , AC k , AD W k and AD V k represent the associated uncertainties. 
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The only assumptions we made on the uncertainties is that they are additive and are within 
a known set. In order to proceed the analysis it is necessary more information about the 
uncertainties. Usually the uncertainties are assumed norm bounded or within a polytope. The 
second approach requires more complex analysis, although the norm bounded set is within 
the set represented by a polytope. 

In this chapter, it will be considered norm bounded uncertainties. For the general case, each 
uncertainty of the system can be represented as 

(26) 
(27) 
(28) 
(29) 
(30) 
(31) 
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where H Ak , H BwJc , H BvJc , H CJc , H Dwk , H Dvk , G X/k , G W/k and G V/k are known. The matrices 
^A ki ^BiuX' FbvJc/ Fq k> ^Dw,k an d ^Dv,k are unknown, time varying and norm-bounded, i.e., 
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These uncertainties can also be represented at a matrix format as 
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However, there is another way to represent distinct uncertainties for each matrix by the 
appropriate choice of the matrices H as follows 
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(34) 
(35) 
(36) 
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where the matrices F 



x,k/ *~w,k 



and F vk of dimensions r 



unknown and norm-bounded, Vfc £ [0, N], i.e., 



x,k x s x,ki r w,k x s w,k' r v,k x s v,k 



Rewriting the uncertainties into a matrix structure, we obtain 
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Our goal is to design a finite horizon robust predictor for state estimation of the uncertain 
system described by (24)-(37). We consider predictors with the following structure 

£ 0|-1 = *0/ (39) 

*k+i\k = ®A\k-i + B w,k w k + B Vjk v k + K k (y k - Qi^jh - D Wrk w k - D vA v k J . (40) 

The predictor is intended to ensure an upper limit in the error estimation variance. In other 
words, we seek a sequence of non-negative definite matrices < P k+ \\ k \ that, for all allowed 
uncertainties, satisfy for each k 



'{ e k+l\k} < Pk+l\k> 



(41) 



where e k+ i\ k = x k+ i — ?(t+i]; c - The matrices <£> k and K k are time-varying and shall be 
determined in such way that the upper bound P k+ \\ k is minimized. 

4.2 A robust estimation solution 

At this part, we shall choose an augmented state vector. There are normally found two options 
are found in the literature: 
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One can note that there is a similarity transformation between both vectors. This 
transformation matrix and its inverse are given by 
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Using the system definition (24)-(25) and the structure of the estimator in (40) then we define 
an augmented system as 
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where 
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Consider P k+ \\ k = cov{x k+ \\. The next lemma give us an upper bound for the covariance 
matrix of the augmented system (44). 

Lemma 1. An upper limit for the covariance matrix of the augmented system (44) is given by 
PqI-i = diag{XQ,0} and 
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(47) 
(48) 
(49) 



Proof: Since x k , w k and v k are uncorrelated signals, and using (8), (9), (39) and (44), it is 
straightforward that Pq|-1 = ^ fl £{Xo,0} and 
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Choose scaling parameters oC\, oc~\ and oC\ satisfying (47)-(49). Using Lemma 2 of Wang 

et al. (1999) and Lemma 3.2 of Theodor & Shaked (1996), we have that the sequence < P k +i\ k \ 

given by (46) is such that P k+ \\ k < P k +\\ k for all instants k. QED. 

Replacing the augmented matrices (45) into (46), the upper bound P k+ \\ k can be partitioned as 
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where, using the definitions presented in Step 1 of Table 2, we obtain 
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,k n Bv,k' 


(57) 


G l,k { U x]c I ~ G x,k F U,k\k-l G I,k) 


-i 

G, 


■,k> 




(58) 


Pn,k\k-1 + P\l,k\k-\ M kPl\,k\k-\> 








(59) 


P\2,k\k-1 + Pu,k\k-l M kPl2,k\k-l> 








(60) 


P 22,k\k-1 + Pl2,k\k-l M kPl2,k\k-l> 








(61) 


P\lc,k - Pl2c,k, 








(62) 


P\2c,k - P22c,k* 








(63) 


S l,k ~ S 2,k- 










(64) 



Since P k+ \\ k > P k +\\ k > 0, Vfc, it is clear that if we define 



k+l\k 



[I 



I]P 



k+l\k 



[' -I] 



(65) 



then we have that P k+ \\ k is an upper bound of the error variance on the state estimation. 
Using the definitions (50) and (65), the initial condition for P 
can be written as 



ifc+l|Jt is P(S\-1 



Xn and P,. 



k+l\k 



Pk+i\k = (At - K k C k ) P u ,c (A k - K k C k ) T - (A k - K k C k ) P 12 , c (4>, c - K k C k f 
- (*t - K k C k ) P\ 2fi (A k - K k C k ) T + (* t - K k C k ) P 22A (*t - K k C k ) T 
+ { B w,k - K k D wk ) W Cik [B Wik - K k D wk ) 



B 



v,k ■ 



K k D V/k ) V Cik 



J v,k' 



K k D 



k u v,k) 



a x k \ H Ak ~ K kHc,k 



3 

w,k 



iT 



a wi \ H Bw,k 



K vS ( H Bv,k 



HA.k -jfc Hc,k) 

K k H Dwk ) (H Bw /k - K k H Dwk ) 
K k H Dvk ) (H Bvk - K k H Dvk ) . 



(66) 
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Note that Pfc+m given by (66) satisfies (41) for any <J>/ C and K k . In this sense, we can choose 

them to minimize the covariance of the estimation error given by P k +i\ k - ^ e calculate the first 
order partial derivatives of (66) with respect to <E>^ and K k and making them equal to zero, i.e., 

3 ^ 



3*T k+l \ k 

d - 
dK k k+l \ k 







0. 



Then the optimal values <£> k = <J>£ and K k = K k are given by 
Kt 



ft 



where 



U 



A kSkC k +Y\,k) [CkS k C k +^2,k 



(A k - K* k C k ) [PnckPiick - ' 



P\\c,k P 12c,k P 22c,k P 12c,k' 

B w , k W Crk Dl k + B Vjk V Cjk Dl k + A u , 



21 



T 



D lD , k Wc,kD w , k 



V v ,kVc,kKk + A 2, 



(67) 
(68) 

(69) 
(70) 

(71) 
(72) 
(73) 



Actually <t>£ and Kt provide the global minimum of Pyt+llJc- This can be proved though the 
convexity of P k+ \\ k at (66). We first have that P k+ \\ k > 0, W k > and V k > 0,Vfc. Then we 
calculate the Hessian matrix to conclude that we have the global minimum: 



HelP 



k+l\k 



* P 



32 P. 



32^ ' k+l\k d 2 [<S> k ,K k ] L k+l\k 
d 2 p d 2 p 

d 2 [K t ,t> k } 1 k+l\k 32Ktk+l\k 



2P 22,k\k-l 2 QS 2 ,; C 

2S 2,/t c t C k S k C k +Y 3/fc 



>0. 



At the previous equations we used the pseudo-inverse instead of the simple matrix inverse. 
Taking a look at the initial conditions P120I-I = ^rzol-l = ^22 01— 1 = ®' one can n °te that 
^22,0 = and, as consequence, the inverse does not exist for all instant k. However, it can be 
proved that the pseudo-inverse does exist. 
Replacing (70) and (69) in (52) and (53), we obtain 



12,k+l\k 



oT 

- 121+1 1 it 



[ 22,/c+l|Jt 



AuP- 



5-1 nT 



k 1 12cM 22cl J \2c,k A k 



At + (A k S k C' +Y U (C k S k Cl + Y 2 , 



A k S k Cl+W 



H 



(74) 



Since (74) holds for any symmetric P k+ \\ k , if we start with a matrix P n+ \\ n satisfying P 12 n+l\n 
^t? +1 1 = ^22 n+l\n f° r some n > 0, then we can conclude that (74) is valid for any k > n. 
The equality allows us some simplifications. The first one is 



S k = P c ,k\k-\ '■= P k\k-1 + P k\k-l G x,k i^Xrk 1 ~ 



x,k l k\k 



l G x,k) G x, 



fc J fc|fc-l- 



(75) 



In fact, the covariance matrix of the estimation error presents a modified notation to deal with 
the uncertain system. At this point, we can conclude that ot xk shall now satisfy 



^l 1 - G x , k Pk\k-iGl k > 0. 



(76) 
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Using (74), we can simplify the expressions for <t>,* Kt and P k+ \\ k - We can define <£> k given in 
Step 4 of Table 2 as <& k = <t>£ . The simplified expression for the predictor gain is given by 



K 



k 



A k P c,k\k-l C k + Y UJ [ C kPc,k\k-l C k +^2,k 



which can be rewritten as presented in Step 4 of Table 2. The expression for the Riccati 
equation can be written as 

P k+l\k = ( A k - KjtQt) P c,k\k-1 ( A k ~ K k C k) 

+ {B m ,k - K-k D u>,k) W c,k {B w ,k - K k D Wik ) 
+ { B v,k - K k D v,k) V c,k { B v,k ~ K k D v,k) 

+ a x,k i H A,k - KlH ck ) {H A:k - K k H c ^) 

+ a w,k i H Bw,k - K k H Dwk ) (H BW:k - K k H DWik ) 

+ a v,k i H Bv,k - K k H Dv,k) {^Bv,k ~ K k H Dv,k) ■ 

Replacing the expression for Kj* in P k +\\ k , we obtain the Riccati equation given in Step 5 of 

Table 2. 

Using an alternative representation, remember the predictor structure: 



*k+i\k = *it%-i + B wJ ,w k + B vk v k + K k (y k - C k x k \ k _ x - D W)k w k - D Vik v k ) . (77) 
Replace <E>£ into (77) to obtain 

%+i|fc = A ck*k\k-i + B w,k™k + B Vlk v k + K k (y k - C c>fc x fc|fc _ 1 - D w>k w k - D v>k v k J , (78) 
where 

Ac* : = A k + A k p k\k-iGl, k {oc'll - G x>k P k \ k _ x G r x ^ ~* G xJu (79) 

C c ,k '■= C k + C k P k \ k _iG X/k [oc Xik I - Gx,kPk\k-iG Xrk ) G Xjk . (80) 

Once again, it is possible to obtain the classic estimator from the structure (79)-(80) for a system 
without uncertainties. 

5. Numerical example 

At this section we perform a simulation to illustrate to importance to consider the 
uncertainties at your predictor design. 

One good way to quantify the performance of the estimator would be using its real variance 
to the error estimation. However, this is difficult to obtain from the response of the 
model. For this reason, we approximate the real variance of the estimation error using the 
ensemble-average (see Ishihara et al. (2006) and Sayed (2001)) given by: 



-i lv 2 
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Step (Initial conditions): ?q|-1 = x an d PqI—i = Xg. 

Step 1: Obtain scalar parameters 0L xk , a w,k an d & v ,k that satisfy (76), 
(48) and (49), respectively. Then define 



A 3,lc 



r 



K xJc H A,k H C* + a w,k H Biv,kHl wJ( + \ k H Bv ,kHbv,k' 

a xi H C,Jc^ fc + <*~lH Dw>k Hl wk + ^ k H Dl , tk Hl vk , 

a xl H Ak H l,k + a 7i,!k H Bw,k H I W/k + a vl H Bv,k H L,k- 



Step 2: Calculate the corrections due to the presence of uncertainties 

P c,k\k-1 '■= P k\k-1 + P k\k-l G x,k [fxl 1 ~ G x,k P k\k-l G x,k) G x,k P k\k-l' 
W c . k := W k + W k Gl , (x-\l - G mi W k Gl .) _1 G wk W k . 



c,k ■= "Vjfc t" ^k^ w ,k {"-wjc 1 ~ ^wM y n^w^) ^w,k^k- 

V c ,k ■= V k + V k Gl k («-jZ - G Vik V k Gl k ) ~ G Vtk V k/ 

Step 3: Define the augmented matrices 

B k ■= [ Biu,k %v,k } , D k ■= [ D W/k D Vik } , U cA := diag { W C/k , V Cik ] . 

Step 4: Calculate the parameters of the predictor as 

K k = (Ak p c,k\k-l C k + BkUckD^ + A U ) (CtP^i^Cj + D k U c , k D T k + A 2//c ) + , 
®k = A k + (A k - K k C k ) p k]k ^Gl k (ct£l - G^Ptit-iGjJ ~* G X/k . 

Step 5: Update {% +1 |fc} and {P fc+ i| fc } as 

*jfc+l|ifc = ®k x k\k-i + B Wrk w k + B Vjk v k + K k [y k - C fc x fc | fc _ 1 - D wM w k - D V/k v k J , 
p k+i\k = A k p c ,k\k-i A k + B kU c ,kB k + A 3;J: 

A k p c,k\k-i C I + A u) { G k p c,k\k-1 G I + DkUc, k Dl + A u ) + [A k P cMk _ x Cl + A 



Table 2. The Enhanced Robust Predictor. 

where e , is the 2-th component of the estimation error vector ej of the realization j defined 
as 

e k ■ x k X k\k-V W 

Another way to quantify the performance of the estimation is though covariance ellipses. The 
use of covariance ellipses allows us to visualize the variance and the cross covariance of a 
system with two states. 
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Consider the benchmark model, used for instance in Fu et al. (2001) and Theodor & Shaked 
(1996), where we added uncertainties in order to affect every matrix of the system, 



Ilk 







h,k 



-0.5 
1 + 038 X)k 



Xk 



1 + 0.1<5 



w,k 



U>h 



[ -100 + 5S Xrk 10 + 1.5S Xik } x k + lOO^z^, 



where 5 n t varies uniformly at each step on the unit interval for n = x,w,v. We also use 
w k = 0.1, V k = 0.9, W k = 0.1 and V k = 2 with initial conditions x = [2 1] T and X = 0.1/. 
The matrices associated to the uncertainties are given by 



H 



A,k 



H 



Bv,k 



HBw,k ~ 

H Cik = 50, H DWik = 0, H DVik = 100, 
G Xik = [0.1 0.03], G^ = 0.01, G Vjk 

The scalar parameters are calculated at each step as 



0C~l = O-max {Gx,k~Pk\k-lGlk} + e *' 
0C~\ = O-max \G Wjk W k Gl k } +£ W , 
0C~l = (Tmax {g^I^G^} + £ v , 



(84) 

(85) 
(86) 
(87) 



where cr max {•} indicates the maximum singular value of a matrix. Numerical simulations 
show that, in general, smaller values of e x , £w and e v result in better bounds. However, this 
can lead to bad inverses calculation. In this example, we have chosen e x = e w = e v = 0.1. 
The mean value of the covariance matrices obtained over 500 experiments at k = 1500 for the 
robust predictor and the classic predictor are 



robust 



14.4 -22.7 
-22.7 76.4 



,P, 



classic 



3.6 -0.6 
-0.6 0.1 



Fig. 2 shows the time evolution of the mean value (over 500 experiments) of both states and 
of their estimated values using the classic and the robust predictors.. It can be verified that 
the estimates of the classic predictor keep oscillating while the robust predictor reaches an 
approximate stationary value. The dynamics of the actual model also presents approximate 
stationary values for both state. It means that the robust predictor were able to better estimate 
the dynamics of the model. 

The covariance ellipses obtained from both predictors and the actually obtained states at 
k = 1500 are shown at Fig. 3. Although the size of the ellipse is smaller for the classic 
Kalman predictor, some states of the actual model are outside this bound. Fig. 4 presents 
the time evolution of the error variances for both states of the system. The error variances 
were approximated using the ensemble-average, defined in Sayed (2001). 
The proposed filter reaches their approximate stationary states after a few steps while the 
Kalman filter did not. Fig. 4 also shows that the actual error variance of the proposed filter 
is always below its upper bound. Although the error variance of the Kalman filter is lower 
than the upper bound of the robust estimator, the actual error variance of the Kalman filter is 
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above its error variance prediction, i.e., the Kalman filter does not guarantee the containment 
of the true signal i/j-- This is a known result and it is presented in Ghaoui & Clafiore (2001). 
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Fig. 2. Evolution of state 2 and its robust estimates. 
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Fig. 3. Mean covariance ellipses after 1500 experiments. 

A comparison with using the robust predictor presented here and another predictor found 
in the literature is shown at ?????. The results presented therein show that the enhanced 
predictor presented here provides a less conservative design, with lower upper bound and 
lower experimental value of the error variance. 



6. Conclusions 

This chapter presented how to design robust predictor for linear systems with norm-bounded 
and time-varying uncertainties in their matrices. The design is based on a guaranteed cost 
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Stats 1 




Fig. 4. Error variances for uncorrelated noise simulation. 

approach using the Riccati equation. The obtained estimator is is capable of dealing with 
systems that present correlated dynamical and measurement noises with unknown mean and 
variance. In most of real life applications this is a common situation. It is also remarkable that 
the separated structure for the noises allows the estimator to have a less conservative upper 
bound for the covariance of the estimation error. 

Further studies may include the use of approach of this chapter to design estimators for 
infinite time horizon discrete systems. Future studies may also investigate the feasibility to 
design a estimator for a more general description of systems: the descriptor systems. 
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1. Introduction 

It is well known that, for the design of tracking control systems, preview information of 
reference signals is very useful for improving performance of the systems, and recently 
much work has been done for preview control systems [Cohen & Shaked (1997); Gershon et 
al. (2004a); Gershon et al. (2004b); Nakura (2008a); Nakura (2008b); Nakura (2008c); Nakura 
(2008d); Nakura (2008e); Nakura (2009); Nakura (2010); Sawada (2008); Shaked & Souza 
(1995); Takaba (2000)]. Especially, in order to design tracking control systems for a class of 
systems with rapid or abrupt changes, it is effective in improving the tracking performance 
to construct tracking control systems considering future information of reference signals. 
Shaked et al. have constructed the Hco tracking control theory with preview for continuous- 
and discrete-time linear time-varying systems by a game theoretic approach [Cohen & 
Shaked (1997); Shaked & Souza (1995)]. Recently the author has extended their theory to 
linear impulsive systems [Nakura (2008b); Nakura (2008c)]. It is also very important to 
consider the effects of stochastic noise or uncertainties for tracking control systems. By 
Gershon et al., the theory of stochastic Hco tracking with preview has been presented for 
linear continuous- and discrete- time systems [Gershon et al. (2004a); Gershon et al. (2004b)]. 
The Hco tracking theory by the game theoretic approach can be restricted to the optimal or 
stochastic optimal tracking theory and also extended to the stochastic Hco tracking control 
theory. While some command generators of reference signals are needed in the papers 
[Sawada (2008); Takaba (2000)], a priori knowledge of any dynamic models for reference 
signals is not assumed on the game theoretic approach. Also notice that all these works have 
been studied for the systems with no mode transitions, i.e., the single mode systems. 
Tracking problems with preview for systems with some mode transitions are also very 
important issues to research. 

Markovian jump systems [Boukas (2006); Costa & Tuesta (2003); Costa et al. (2005); Dragan 
& Morozan (2004); Fragoso (1989); Fragoso (1995); Lee & Khargonekar (2008); Mariton 
(1990); Souza & Fragoso (1993); Sworder (1969); Sworder (1972)] have abrupt random mode 
changes in their dynamics. The mode changes follow Markov processes. Such systems may 
be found in the area of mechanical systems, power systems, manufacturing systems, 
communications, aerospace systems, financial engineering and so on. Such systems are 
classified into continuous-time [Boukas (2006); Dragan & Morozan (2004); Mariton (1990); 
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Souza & Fragoso (1993); Sworder (1969); Sworder (1972)] and discrete-time [Costa & Tuesta 
(2003); Costa et al. (2005); Lee & Khargonekar (2008); Fragoso (1989); Fragoso et al. (1995)] 
systems. The optimal, stochastic optimal and Hco control theory has been presented for each 
of these systems respectively [Costa & Tuesta (2003); Fragoso (1989); Fragoso et al. (1995); 
Souza & Fragoso (1993); Sworder (1969); Sworder (1972)]. The stochastic LQ and Hco control 
theory for Markovian jump systems are of high practice. For example, these theories are 
applied to the solar energy system, the underactuated manipulator system and so on [Costa 
et al. (2005)]. Although preview compensation for hybrid systems including the Markovian 
jump systems is very effective for improving the system performance, the preview tracking 
theory for the Markovian jump systems had not been yet constructed. Recently the author 
has presented the stochastic LQ and Hco preview tracking theories by state feedback for 
linear continuous-time Markovian jump systems [Nakura (2008d) Nakura (2008e); Nakura 
(2009)], which are the first theories of the preview tracking control for the Markovian jump 
systems. For the discrete-time Markovian jump systems, he has presented the stochastic LQ 
preview tracking theory only by state feedback [Nakura (2010)]. The stochastic LQ preview 
tracking problem for them by output feedback has not been yet fully investigated. 
In this paper we study the stochastic optimal tracking problems with preview by state 
feedback and output feedback for linear discrete-time Markovian jump systems on the finite 
time interval and derive the forms of the preview compensator dynamics. In this paper it is 
assumed that the modes are fully observable in the whole time interval. We consider three 
different tracking problems according to the structures of preview information and give the 
control strategies for them respectively. The output feedback dynamic controller is given by 
using solutions of two types of coupled Riccati difference equations. Feedback controller 
gains are designed by using one type of coupled Riccati difference equations with terminal 
conditions, which give the necessary and sufficient conditions for the solvability of the 
stochastic optimal tracking problem with preview by state feedback, and filter gains are 
designed by using another type of coupled Riccati difference equations with initial 
conditions. Correspondingly compensators introducing future information are coupled with 
each other. This is our very important point in this paper. Finally we consider numerical 
examples and verily the effectiveness of the preview tracking theory presented in this paper. 
The organization of this paper is as follows: In section 2 we describe the systems and 
problem formulation. In section 3 we present the solution of the stochastic optimal preview 
tracking problems over the finite time interval by state feedback. In section 4 we consider 
the output feedback problems. In section 5 we consider numerical examples and verify the 
effectiveness of the stochastic optimal preview tracking design theory. In the appendices we 
present the proof of the proposition, which gives the necessary and sufficient conditions of 
the solvability for the stochastic optimal preview tracking problems by state feedback, and 
the orthogonal property of the variable of the error system and that of the output feedback 
controller, which plays the important role to solve the output feedback problems. 
Notations: Throughout this paper the superscript ' stands for the matrix transposition, | ■ | 
denotes the Euclidean vector norm and | v | R also denotes the weighted norm v'Rv. O 
denotes the matrix with all zero components. 

2. Problem formulation 

Let (Q,, F, P) be a probability space and, on this space, consider the following linear discrete- 
time time-varying system with reference signal and Markovian mode transitions. 
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x(k+l) = A dm(k) (k)x(k) + G dm(k) (k)« d (k) + B 2d m(k) (k)u d (k) + B 3d m(k) (k)r d (k) 

Z d ( k ) = C ld,m(k) ( k ) X ( k ) + D 12d,m(k) ( k ) U d ( k ) + D 13d,m(k) ( k ) r d ( k ) CO 

y( k ) = C 2d,m(k)( k ) X ( k ) + H d,m(k)( k )' y d( k ) 

x(0) = x m(0) = i 

where xe R is the state, ff>d e B: is the exogenous random noise, u d e R m is the control 

input, Zd e R is the controlled output, rd(-) e R is known or measurable reference signal 

and ye R is the measured output, xo is an unknown initial state and io is a given initial 

mode. 

Let M be an integer and {m(k)} is a Markov process taking values on the finite set 

cp={l,2, -,M) with the following transition probabilities: 

P{m(k+l)=j|m(k)=i}:=p d , ij (k) 

where pd,ij(k)>0 is also the transition rate at the jump instant from the mode i to j, i =£ j, and 
V p ..(Jc) = 1 ■ Let Pd(k) =[ pd,ij(k)] be the transition probability matrix. We assume that all 

these matrices are of compatible dimensions. Throughout this paper the dependence of the 
matrices on k will be omitted for the sake of notational simplicity. 
For this system (1), we assume the following conditions: 
Al: Di2d, m (k)(k) is of full column rank. 

A2: Dl2d,m(k)'( k )Cld,m(k)(k)=0, Dl2d,m(k)'( k )Dl3d,m(k)(k)=0 

A3: E{x(0)!=u , E{ffl d (k)}=0, 
E{cDd(k)cDd l (k)l (m (k)=i)}=X i , 
E{x(0)x'(0)l {m(0) -i )}=Q io (0), 
E{od(0)x'(0)l{m(0)-i }}=O, 

E{cD d (k)x ' (k)l {m(k)=i}} =0, 

E{Q)d(k)ud'(k)l( m (k)=i|}=0, 
E{ffld(k)rd l (k)l |m(k )= i )}=0 

where E is the expectation with respect to m(k), and the indicator function l| m (k)=i):=l if 
m(k)=i, and l{m(k)-i}: = if m(k)7 lt i. 

The stochastic optimal tracking problems we address in this section for the system (1) are to 
design control laws Ud() e l2[0,N-l] over the finite horizon [0,N], using the information 
available on the known part of the reference signal rd() and minimizing the sum of the 
energy of Zd(k), for the given initial mode io and the given distribution of Xo. Considering the 
stochastic mode transitions and the average of the performance indices over the statistical 
information of the unknown part of rd, we define the following performance index. 



JdN( X 0, U d. r d) :=E Z%{l C ld,m(k)( k ) X ( k ) + D 13d,m(k)( k ) r d( k )l 
*:=0 

+ E\{ |D 12d,m(k)( k )Ud( k )l 



(2) 



E^- means the expectation over Rk+h , h is the preview length of rd(k), and Rk denotes the 
future information on rd at the current time k, i.e., Rk :={rd(l); k<l<N}. This introduction of 
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E^- means that the unknown part of the reference signal follows a stochastic process, 
whose distribution is allowed to be unknown. 

Now we formulate the following optimal fixed-preview tracking problems for the system (1) 
and the performance index (2). In these problems, it is assumed that, at the current time k, 
rd(l) is known for 1 < min(N, k+h), where h is the preview length. 
The Stochastic Optimal Fixed-Preview Tracking Problem by State Feedback: 
Consider the system (1) and the performance index (2), and assume the conditions Al, A2 
and A3. Then, find u d minimizing the performance index (2) where the control strategy u d 
(k), < k < N-l, is based on the information Rk+h:={rd(l); < 1 < k+h} with < h < N and the 
state information Xj<:={x(l); < 1 < k). 

The Stochastic Optimal Fixed-Preview Tracking Problem by Output Feedback: 
Consider the system (1) and the performance index (2), and assume the conditions Al, A2 
and A3. Then, find u d minimizing the performance index (2) where the control strategy u d 
(k), < k < N-l, is based on the information Rk+h:={rd(l); < 1 < k+h} with < h < N and the 
observed information Yk:={y(l); < 1 < k}. 

Notice that, on these problems, at the current time k to decide the control strategies, Rk+h can 
include any noncausal information in the meaning of that it is allowed that the future 
information of the reference signals jrd(l); k < 1 < k+h} is inputted to the feedback controllers. 

3. Design of tracking controllers by state feedback 

In this section we consider the state feedback problems. 

Now we consider the coupled Riccati difference equations [Costa et al. (2005); Fragoso 

(1989)] 

Xi(k)=Ad,i'(k)Ei(X(k+l),k)Ad,i(k)+Cid/Cid,i-F 2 /T2,iF2,i(k), k=0, 1, - (3) 

where E 1 (X(k+l),k)= XjfiP^W X j+1 (k+l), X(k)=(Xa(k), -, X M (k)), 

T 2 ,i (k) = D 12d /D 12d4 + B M /Ei (X(k + l),k)B 2d4 , 

R 2 ,i( k ) = B 2d,i' E i( X ( k + 1 )' k ) A d,i' 

F2,i(k) = -T 2 > 2 ,,(k) 
and the following scalar coupled difference equations. 

a, (k)=Ei( a (k+l),k)+tr{G d ,iX i G d ,i'E 1 (X(k+l),k)} (4) 

where Ei( a (k+l),k)= Y™ =lVdiij {k), «, (k+1) and « (k)=( a, (k), ... , a M (k)). 

Remark 3.1 Note that these coupled Riccati difference equations (3) are the same as those for 
the standard stochastic linear quadratic (LQ) optimization problem of linear discrete-time 
Markovian jump systems without considering any exogeneous reference signals nor any 
preview information [Costa et al. (2005); Fragoso (1989)]. Also notice that the form of the 
equation (4) is different from [Costa et al. (2005); Fragoso (1989)] in the points that the 
solution a (•) does not depend on any modes in [Costa et al. (2005)] and the noise matrix Gd 
does not depend on any modes in [Fragoso (1989)]. 
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We obtain the following necessary and sufficient conditions for the solvability of the 
stochastic optimal fixed-preview tracking problem by state feedback and an optimal control 
strategy for it. 

Theorem 3.1 Consider the system (1) and the performance index (2). Suppose Al, A2 and 
A3. Then the Stochastic Optimal Fixed-Preview Tracking Problem by State Feedback for (1) 
and (2) is solvable if and only if there exist matrices Xi(k)>0 and scalar functions a ; (k), i=l, 
-,M, satisfying the conditions Xi(N)=Cid,i'(N)Cid,i(N) and a, (N)=0 such that the coupled 
Riccati equations (3) and the coupled scalar equations (4) hold over [0,N]. Moreover an 
optimal control strategy for the tracking problem (1) and (2) is given by 

u d (k)=F 2 , i (k)x(k)+D u , 1 (k)r d (k)+D eu , i (k)E i ( C (k+l),k) for i=l, -,M 

where D U4 (k)=- T 2 } (k)B 2 d/Ei(X(k+l),k)B 3 d,i and D eu ,i(k)=- T 2 ] (k)B 2d ,i'. { (k), i=l, -,M, 
k e [0,N] satisfies 



i (k) = A d / (k)E ; (0(k + l),k) + B d ,i (k)r d (k), 
^(N) = C ld /D 13d/i r d (N) 



(5) 



where E,( (k+l),k)= ^Pd.ijW &, ( k+1 ) and d ( k H °l ( k )' "v &m ( k ))> 

A d /(k) = A d , I -D eu /T 2/i F 2i (k), 

Bi.i (k) = A d / E s (X(k + l),k)B 3d4 - F 2 /T 2ii D U/i (k) + C ld /D 13d4 

and ci (k) is the 'causal' part of i (•) at time k. This ci is the expected value of t over Rk 
and given by 

ci (l) = A d /(l)E i {0 c (l + l),l) + B d ,i(l)r d (l), k + l<l<k + h, 

6» c ,(k + h + l) = 0ifk + h<N-l (6) 

C/) ,(k + h + 1) = C ld /D 13d-i r d (N), k + h = N 

where E,( C (k+l),k)= X^P^W c,, ( k+ l) and £ c (k)=( ^ (k) , ..., cM (k)). 
Moreover, the optimal value of the performance index is 



7 dN (xO, M ;,r d ) = tr{Q (o X Io } + « io (0) + E{E So {2^ o 'x }} 

+E { Z %( I ^/ijD^w ( k )E m(k) (C (k + l),k) | 2 }} + U (r d ) 



(7) 



where 0" mW (k)= « m(i) (k)- C/(n()t) (k), k s [0,N], 

E,(6f (k+l),k)=£j a p^(fc) 0", (k+l), (?; (k)=(^ (k), -, 6T M (k)) and 



L('' d ) = E{E^{|D 1 3 d , m(N) (N)r d (N)| 2 }} + EJ £ %{-ltm(/c)D^ m(k )(k)E m{k) (0(k +l),k)| 2 

-2E m(k) (^'(k+l),k)D^ m(k) 'T 2 , m(fe) D U/m(k) (k)r d (k) 

+ 2E m(k) (^(k+l),k)B 3d , m(k) (k)r d (k)+J d , k>m(k) (r d )}}, 
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Jd,k,m(k) ( r d) = r d'(k)[-D U; m(k)'72 /m (ic)E ) u,m(k) (k) + B 3d _ m ( k )'E m ( k ) (X(k + l),k}B 3d m(k j 
+ E ) i3d,m(k)'E ) 13d,m(k)J r d(k)- 

(Proof) See the appendix 1. 

Remark 3.2 Note that each dynamics (6) of C , , which composes the compensator 

introducing the preview information, is coupled with the others. It corresponds to the 

characteristic that the Riccati difference equations (3) are coupled with each other, which 

give the necessary and sufficient conditions for the solvability of the stochastic optimal 

tracking problem by state feedback. 

Next we consider the following two extreme cases according to the information structures 

(preview lengths) of rd: 

i. Stochastic Optimal Tracking of Causal jrd(-)}: 

In this case, jrd(k)} is measured on-line, i.e., at time k, rd(l) is known only for l<k. 

ii. Stochastic Optimal Tracking of Noncausal {rd(-)|: 

In this case, the signal jrd(k)} is assumed to be known a priori for the whole time interval 

ke[0,N]. 

Utilizing the optimal control strategy for the stochastic optimal tracking problem in 

Theorem 3.1, we present the solutions to these two extreme cases. 

Corollary 3.1 Consider the system (1) and the performance index (2). Suppose Al, A2 and 

A3. Then each of the stochastic optimal tracking problems for (1) and (2) is solvable by state 

feedback if and only if there exist matrices Xi(k) >0 and scalar functions a, (k), i=l, —,M, 

satisfying the conditions Xi(N)=Cid,i'(N)Cid,i(N) and a t (N)=0 such that the coupled Riccati 

difference equations (3) and the coupled scalar equations (4) hold over [0,N]. Moreover, the 

following results hold using the three types of gains 

K d , x ,i(k)=F 2 ,i(k), K rd ,i(k)=D u ,i(k) and Kd,e,i(k)=D eu ,i(k) for i=l, -,M. 
i. The control law for the Stochastic Optimal Tracking of Causal {rd(-)} is 

Ud,si(k)=K d ,x,i(k)x(k)+K rd ,i(k)rd(k) for i=l, -,M 
and the value of the performance index is 

Jd N (xo, u d ,si, r d )=tr{ Q, o X, o }+ a, o (0)+E{ E^ {2 ^ 'x }| 

N-l 

+E{ X % { I T% {k) D 8u , m(k )(k)E m(k) ( (k +l),k) | 2 }}+ ] d ( r d ). 

ii. The control law for the Stochastic Optimal Tracking of Noncausal jrd(-)} is 

u d , S 2(k)=Kd,x, 1 (k)x(k)+K rd , 1 (k)rd(k)+Kd,B,i(k)E i ( (k +l),k) for i=l, -,M 
with { (■) given by (5) and the value of the performance index is 

JdN(xo,Ud,s2,r d )=tr{Q, o X, q }+ a, o (0)+2^ >+ J, (r d ). 

(Proof) 

i. In this causal case, the control law is not affected by the effects of any preview 
information and so C (k)=0 for all k s [0,N] since the each dynamics of C , becomes 
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autonomous. As a result we obtain (k)= 6~ (k) for all ks [0,N]. Therefore we obtain 
the value of the performance index JdN(xo, Ud, s i,rd)- 
ii. In this noncausal case, h=N-k and (5) and (6) becomes identical. As a result we obtain 
(k)= e c (k) for all k s [0,N]. Therefore we obtain 9~ c (k)=0 for all k s [0,N] and the value 
of the performance index JdN(xo, u is2, id)- Notice that, in this case, we can obtain the 
deterministic value of i (0) using the information of jrd()} until the final time N and so 
the term E{ E^- {2 f 'xo}} in the right hand side of (7) reduces to 2 9 { 'uo. (Q.E.D.) 

4. Output feedback case 

In this section, we consider the output feedback problems. 
We first assume the following conditions: 

A4: G d , m (k)(k)Hd, m( k) l (k)=0, H d , m (k)(k)Hd, m ( k ) l (k)>0 
By the transformation 

ud, c (k):=u d(k)-D u ,i(k)r d (k)-Deu,i(k)Ei( C (k+l),k) 

and the coupled difference equations (3) and (4), we can rewrite the performance index as 
follows: 

Jd N ( x o, M ^,r d ) = MQ, X !o } +« !o (0) 

+E |Z E R7{|^( k )- F 2, m( k)(k)x(k)-D^ m(k) (k)E m(k) (^(k + l),k)|| 

+L(r d ) 
and the dynamics can be written as follows: 

x(k+l)=Ad,m(k)(k)x(k)+G d ,m(k)(k)cDd(k)+B2d,m(k)(k) Ud,c (k)+ Td,c (k) 

where 

rd,c (k)=B2d,m(k){D u , m (k)(k)r d (k)+Deu,m(k)(k)E m ( k )( C (k+l),k)j+B 3 d,m(k)(k)r d (k). 
For this plant dynamics, consider the controller 

X e (k + 1) = A dm(k) (k)x e (k) + B 2d >m(k) (k)u d , c * (k) 

+B 3 d,m(k) {k)rd,c (k) - M m(k) (k)[y (k) - C 2dm(k) f e (k)] 
i t ,(0) = EJE^{x }| = /u , M ( j /C -(k)=F 2/m(k) (k)x e (k) 



(8) 



where M m (k) are the controller gains to decide later, using the solutions of another coupled 
Riccati equations introduced below. 
Define the error variable 

e(k):=x(k)-i e (k) 
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and the error dynamics is as follows: 

e(k+l)=A d ,m(k)(k)e(k)+G d ,m(k)(k)cDd(k)+M m( k)(k)[y(k)-C2d,m(k)x e (k)] 

= [ A d ,m(k)+Mm(k)C2d,m(k)] (k)e(k) + [Gd,m(k)+M m (k)Hd,m(k)] (k)ffld(k) 

Note that this error dynamics does not depend on the exogenous inputs Ud nor rd. Our 
objective is to design the controller gain M m (k) which minimizes 

Jd N (xo, u<t,c* , rd)=tr{ Q, o X, o }+ «, o (0)+E{ E^ {2 ig 'x }| 

N-l 

+E (Z E^{|F 2 , m(k) (k)e(k) 
k=0 

-Deu,m(k)(k)E m(k )( e- (k +l),k) \ 2 TiMt) }}+ ] d ( r d ) 

Notice that e(k) and E m (k)( 9~ (k +l),k) are mutually independent. 

We decide the gain matrices Mi(k), i=l, —,M by designing the LMMSE filter such that 
V E{E=-{|e(k) | }} is minimized. Now we consider the following coupled Riccati 

difference equations and the initial conditions. 



(9) 



Y j( k + 1 ) = I ^[Ad/WA^- A d/i Y i (k)C 2d /(H d4 H d /n i (k) 

is](k) 

+C 2d , i Y I (k)C 2d4 ')- 1 C 2d/i Y i (k)A d / + n 1 (k)G d4 G d4 

Y 1 (0) = n i (0)(Q, o -^ ') 

where 

ni (k):=P{m(k)=i}, ^ViAi (*) "'=% I^i (k) = 1 , J(k):={i 6 N; ni (k)>0}. 

These equations are also called the filtering coupled Riccati difference equations [Costa & 
Tuesta (2003)]. 
Now since 

E{ E^ {e(0)}}=E{ E^ {x }-E{ E^ {x }}}=E{ E^ x }}-u =0 
and n, c (0) is deterministic if r d (l) is known at all 1 e [0,k+h], 

E{ E^ je(0) r d , c 1 (0)l im(0) = i| }}=n i (0)E{ E^ je(0)}} r d , c '(0)=O 
and so we obtain, for each k e [0,N], 

E{ E^ (e(k) r d , c 1 (k)l {m(k) .i)}}=n i (k)E{ E^ (e(k)}} n, c '(k)=0. 

Namely there exist no couplings between e(-) and r d ,c (■). The development of e(-) on time k 
is independent of the development of r d , c (■) on time k. Then we can show the following 
orthogonal property as [Theorem 5.3 in (Costa et al. (2005)) or Theorem 2 in (Costa & Tuesta 
(2003))] by induction on k (See the appendix 2). 
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E{E^{e(k)V(k)l {m(kH1 }}=0. (10) 

Moreover define 

ijtkl-ElE.efkle'Ml^H)) 
and then we can show 

Yi(k)=^(k). 

From all these results (orthogonal properties), as the case of r d (-)=0, using the solutions of 
the coupled difference Riccati equations, it can be shown that the gains M m (k) minimizing JdN 
are decided as follows (cf. [Costa & Tuesta (2003); Costa et al. (2005)]): 



M,(k) = 



-AyY, (k)C 2d /(H d , i H d /n i (k) + C 2d/i Yi (k)C 2d /)~ for i e J(k) 

(11) 

OforieJ(k) 



Finally the following theorem, which gives the solution of the output feedback problem, 

holds. 

Theorem 4.1 Consider the system (1) and the performance index (2). Suppose Al, A2, A3 

and A4. Then an optimal control strategy which, gives the solution of the Stochastic Optimal 

Fixed-Preview Tracking Problem by Output Feedback for (1) and (2) is given by the 

dynamic controller (8) with the gains (11) using the solutions of the two types of the coupled 

Riccati difference equations (3) with X i (N)=Cid,i l (N)Ci d , i (N) and (9) with Yi(0)= ni(0)(Q, o - 

u uo'). 

Remark 4.1 Notice that 

E{ E Tk { | z d (k) | 2 }}= Ylti tr {Cid,iCid/ E{ E^ {x(k)x'(k)l {m(kH) }}} 

+E{ E^ { | D 12 d,m(k)(k)u d (k) | 2 +2x l (k)C ld , i 'D 1 3d,ir d (k)}}. 
Then, with regard to the performance index, the following result holds. 

E{ E^ { | z d (k) | 2 }}=E{ E^ { | 'z e (k) | 2 }}+ Y!ti tr { Cid,iYi(k)C ld /} 

+E f Zfei £ R, {2e l (k)C ld , i 'D 1 3d, 1 r d (k)l im(k) = ll }} 
where 

z e (k)=Cid,m(k) x e (k)+D 12 d,m(k)(k)u d (k)+Di3d, m (k)(k)r d (k) 
and we have used the property 

E{ E^ {x(k)x'(k)l {m{k) . i) }}=E{ E^ {e(k)e 1 (k)l |m(k) =i l }}+E{ E^ {i e (k) x e '(k)l !m(k)=i} }} 
= Y i (k)+E{E^{x e (k)x e , (k)l {m(k) . i) }} 

by the orthogonal property (10). 
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Note that the second and third terms in the right hand side do not depend on the input Ud. 
Then we obtain 



JdN (xo u d r d ) = E(X[%{ I z e (k) | 2 } tlXCujYi (k)C ld /} 

k=0 

+ Z!! 1 % i { 2e, (k)C ld /D 13d , i r d (k) l {m(k)=i} }}] (12) 



+ 1 C ld , m(N) x(N) + D 13d/m(N) r d (N) r 

N 

I 
k-0 



Therefore minimizing (12) is equivalent to minimizing E{ V E„- { | z e (k) | } subject to the 
dynamics 

x e (k+l)=A d ,m(k)(k) x e (k)+B 2d ,m(k)(k) u ilC * (k)+ r d ,c (k) -M m(k) (k)v(k), x e (0)= E^ {x }=u 

where 

v(k)=y(k)-C 2d ,m(k) x e (k) 

and Ud,c* (k) is the state feedback controller with the form 

K d ,x,i(k) x e (k)+K rd4 (k)r d (k)+K d ,e,i(k)Ei( (k +l),k) for some gains K d , x ,i, K rd/i and K dAi . Note 

that the term M m (k)(k)v(k) plays the same role as the "noise" term Gd,m(k)(k)ff>d(k) of the plant 

dynamics in the state feedback case. 

Remark 4.2 As the case of r d ()=0, the separation principle holds in the case of r d ()*0. 

Namely we can design the state feedback gains F2, m (k)(k) and the filter gains M m (k) separately. 

Utilizing the optimal control strategy for the stochastic optimal tracking problem in 

Theorem 4.1, we present the solutions to the two extreme cases. 

Corollary 4.1 Consider the system (1) and the performance index (2). Suppose Al, A2, A3 

and A4. Then optimal control strategies by output feedback for the two extreme cases are as 

follows using the solutions of the two types of the coupled Riccati difference equations (3) 

withX i (N)=Cid, i l (N)Cid,i(N) and (9) with Yi(0)= rn(0)(Q,- -u u '): 

i. The control law by output feedback for the Stochastic Optimal Tracking of Causal jrd(-)} is 

x e (k+l)=A d , m( k)(k) x e (k)+B 2d ,m(k)(k) ««j,i* (k)+ n,\ (k) -M m(k) (k)v(k) 

x e (0)=uo 
M rf/ i(k):=u d (k)-D U/i (k)r d (k) 
Ud,v (k)=F 2 , m (k)(k) x e (k) 
n,\ (k)=B 2 d, m (k)D u , m ( k )(k)rd(k)+B 3 d, m (k)(k)rd(k) 
and the value of the performance index is 

Jd N (x , u ilV , r d )=tr{ Q, q X, o }+ «, q (0)+E{ E^ {2 ^ 'x }} 

N-l 

+E{^ E^{|F 2 , m(k) (k)e(k) 

S:=0 

-D eu , m( k)(k)E m(k) ( 6 (k +l),k) \\ Mk) }}+ /, ( rd ). 
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ii. The control law by output feedback for the Stochastic Optimal Tracking of Noncausal 
{«(•)} is 

x e (k+l)=A d ,m(k)(k) x e (k)+B 2d ,m(k)(k) Ud,2* (k)+ r d ,2 (k)-M m(k) (k)v(k) 

K (0)=uo 
ud.i (k):= u d(k)-Du,i(k)r d (k)-Deu,i(k)Ei( 9 (k+l),k) 
Ui,2* (k)= F 2 , m (k)(k) x e (k) 

rd,2 (k)=B2d,m(k){D u ,m(k)(k)rd(k)+De u ,m(k)(k)E m (k)( # (k+l),k)}+B 3 d,m(k)(k) r d (k) 
and the value of the performance index is 



JdN(xo, a w . , r d )=tr{ Q, o X, o }+ «, o (0)+ 2 ^ 'u +E{ £ E^ { | F 2 , m(k) (k)e(k) l^ }}+/,( r d ) 



N-l 

I 

A-ll 



(Proof) As the state feedback cases, C (k)=0, i.e., 9 (k)= 9 C (k) for all k e [0,N] in the case i), 
and 9 (k)= 6 1 , (k), i.e., ^ (k)=0 for all k e [0,N] in the case ii). 



5. Numerical examples 

In this section, we study numerical examples to demonstrate the effectiveness of the 
presented stochastic LQ preview tracking design theory. 

We consider the following two mode systems and assume that the system parameters are as 
follows, (cf. [Cohen & Shaked (1997); Shaked & Souza (1995)].): 

x(k + l) = A d;m(k) (k)x(k) + G d (k)« d (k) + B 2d (k)u d (k) + B 3dm(k) (k)r d (k) 

x(0) = x m(0) = i m(k) = 1,2 

z d ( k ) = C ldm(k) (k)x(k) + D 12d m(k) (k)u d (k) + D 13d m(k) (k)r d (k) 

y( k ) = C 2d,m(k)( k ) X ( k ) + H d,m(k)( k H( k ) 



where 
Model 

A d ,i 



B 



3d,l" 



Cid,i- 






1 


-0.8 1.6 


1.5" 



/ B3d,2 


"-0.5 0.2 









Mode 2: 

"0 1 

, A d 2= 

[l.6 1.6 

/ Cld,2 



,G d 



" 


, B2d~ 





0.1 




l_lj 



0.5 



0.1" 




, Dl2d= 



0.1 



,Di 



-1.0 





(13) 



Let 



0.3 0.7 
0.6 0.4 



be a stationary transition matrix of (m(k)}. We set xo=col(0,0) and io=l. 
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Then we introduce the following objective function. 



N 

z 

A--0 



Jdw(x o, Ud, r d ):=E{ ^ E^ { | C 1 d, m (k)(k)x(k)+Di3d, m (k)(k)r d (k) | 2 }} 

N-l 

+0.01E{£ E^{|u d (k)| 2 }} 



By the term B3d,i(k)rd(k), i=l,2, the tracking performance can be expected to be improved as 
[Cohen & Shaked (1997); Shaked & Souza (1995)] and so on. The paths of m(k) are generated 
randomly, and the performances are compared under the same condition, that is, the same 
set of the paths so that the performances can be easily compared. 

We consider the whole system (13) with mode transition rate Pd over the time interval 
ke [0,100]. For this system (13) with the rate matrix Pd , we apply the results of the optimal 
tracking design theory by output feedback for rd(k)=0.5sin(nk/20) and rd(k)=0.5sin(nk/100) 
with various step lengths of preview, and show the simulation results for sample paths. 




Time[s] 
Fig. 1(a). r d (k)=0.5sin(nk/20) 




40 50— — __60 70 

Time[s] h = 4 

Fig. 1(b). r d (k)=0.5sin(nk/100) 

Fig. 1. The whole system consisting of mode 1 and mode 2: The errors of tracking for various 
preview lengths 
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It is shown in Fig. 1(a) for r d (k)= 0.5sin(nk/20) and Fig. 1(b) r d (k)=0.5sin(nk/100) that 
increasing the preview steps from h=0 to h=l,2,3,4 improves the tracking performance. In 
fact, the square values | Cid,i(k)x(k) + Di3d(k)rd(k) | of the tracking errors are shown in 
Fig. 1(a) and (b) and it is clear the tracking error decreases as increasing the preview steps 
by these figures. 

6. Conclusion 

In this paper we have studied the stochastic linear quadratic (LQ) optimal tracking control 
theory considering the preview information by state feedback and output feedback for the 
linear discrete-time Markovian jump systems affected by the white noises, which are a class 
of stochastic switching systems, and verified the effectiveness of the design theory by 
numerical examples. In order to solve the output feedback problems, we have introduced 
the LMMSE filters adapted to the effects of preview feedforward compensation. In order to 
design the output feedback controllers, we need the solutions of two types of coupled 
Riccati difference equations, i.e., the ones to decide the state feedback gains and the ones to 
decide the filter gains. These solutions of two types of coupled Riccati difference equations 
can be obtained independently i.e., the separation principle holds. Correspondingly the 
compensators introducing the preview information of the reference signal are coupled with 
each other. This is the very important research result in this paper. 

We have considered both of the cases of full and partial observation. However, in these 
cases, we have considered the situations that the switching modes are observable over 
whole time interval. The construction of the design theory for the case that the switching 
modes are unknown is a very important further research issue. 

Appendix 1. Proof of Proposition 3.1 

(Proof of Proposition 3.1) 

Sufficiency: 

Let Xj(k)>0 and a, , i=l, . . ., M, be solutions to (3) and (4) over [0,N] such that 

X i (N)=C ld4 l (N)C ld , i (N) and a, (N)=0. 

Define 



4t,„ m ■- E^ {E{x'(k+l)X m(k+1) (k+l)x(k+l)+ a m(k+1) (k+l) | x(k),m(k)}} 
- E F (x'(k)X m( k) x(k)+ a m(k)(k)} 

We first consider the case of rd()=0. Then the following equalities hold by the assumptions 
A3. 

E{x'(k+l)X m(k+1) (k+l)x(k+l)+ « m(k+ i)(k+l) | x(k),m(k)} 

= E{ (A d ,m(k)(k) x(k)+Gd,m( k )(k)cDd(k)+B 2 d,m( k )(k)ud(k))' 

x X m(k+ i) (k+1) (A d ,m(k)(k) x(k)+G d ,m( k )(k)cDd(k)+B2d,m( k )(k)ud(k)) 
+ a m( k +i)(k+l) | x(k),m(k)} 
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=(Ad,m(k)(k)x(k)+B 2 d,m(k)(k)ud(k))' 

x E m(k) (X(k+l),k) (A d , m( k)(k) x(k)+B 2 d,m(k)(k)u d (k)) 

+ Xji tr f Gd,i ( k ) x < (k)G d /(k)E 1 (X(k+l),k)}+E{ a m(k+ i) (k+1) | x(k),m(k)} 

It can be shown that the following equality holds, using the system (1) and the coupled 
Riccati equations (3) and the coupled scalar equations (4). ([Costa et al. (2005); Fragoso 
(1989)]) 

4, m(i) = E^ {- 1 z d (k) | 2 + | T% (k) (k)[u d (k)-F 2 , m(k) (k) x(k)] | 2 } 

Moreover, in the genaral case that rd(-) is arbitrary, we have the following equality. 

A, m( k) = % f I Mk) | 2 + 1 T^ {k) (k)[u d (k)-F 2 , m(k) (k)x(k)]-D u , m(k) (k)r d (k) | 2 
+ 2x'(k) Bd, m (k) (k)r d (k)+Jd,k,m(k)(rd)} 

Notice that, in the right hand side of this equality, Jd,k,m(k) (rd), which means the tracking 
error without considering the effect of the preview information, is added. 
Now introducing the vector m , k \ , which can include some preview information of the 
tracking signals, 

%^ { £ f «»(*+!) ' (k+l)x(k+l) | x(k),m(k)}}- E^ { 6 m(k) '(k)x(k)} 

= E^- {E m(k) ( l (k+l),k)(Ad, m (k)(k)x(k)+Gd, m( k)(k)cDd(k) 

+B 2 d, m (k)(k)ud(k)+B3d, m (k)(k)r d (k))}- E^ { m(k) ' (k)x(k)} 

Then we obtain 



■,m(k) 



+2{ %- {E{ m{k+1) ' (k+l)x(k+l) | x(k),m(k)}}- Ejr I m(k) ' (k)x(k)}} 



where 



= E^ {- 1 z d (k) | 2 + | r 2 % } (k)[u d (k)-F 2 , m( k)(k)x(k)-D u , m(k) (k)r d (k)] | 2 

+ 2x'(k) Bd,m(k) (k)r d (k)+J d , k ,m( k ) (rd)} 
+2 E^ {{ E m(k) ( l (k+l),k)(Ad, m ( k )(k)x(k)+Gd, m(k) (k)Q)d(k) (14) 

+B 2d , m ( k )(k)ud(k)+B3d, m(k )(k)rd(k))}- E^ { m{k) '(k)x(k)}} 

= E Tt {- 1 z d (k) | 2 + | T% {k) (k)[u d (k)-F 2 , m( k)(k)x(k)-D u , m(k) (k)r d (k) 

-D eu ,m(k)(k)E m(k) ( (k+l),k)] | 2 + ] d/kMk) (r d )} 



9, (k)= A d ,, '(k)Ei( (k+l),k)+ B d/ i (k)r d (k) 
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to get rid of the mixed terms of r d and x, or 6 m i k \ and x. ] d k m , k) (ra) means the tracking error 
including the preview information vector 9 and can be expressed by 



i2 



W)^" 1 Tl% Deu, m(k) (k)E m(k) (0(k + l),k)] 
-E m (k)( 9 '(k+l),k)De u ,m(k) ' T 2 , m (k)D u , m (k)(k)rd(k) 
+2 E m(k )( 9 '(k+l),k)B3d,m(k)rd(k)+Jd,k,m(k) (ra) 

Taking the sum of the quantities (14) from k=0 to k=N-l and adding E{ | Cid, m (N)(N)x(N)-i 
E>i3d, m (N)(N)rd(N) | } and taking the expectation E{ }, 



N-l 

I 

A-0 



£ E{ E^- { | z d (k) | 2 }}+E{ | C ld , m( N)(N)x(N)+ Di 3d , m(N) (N)r d (N) f 

D 

N-l 

+ Z E{ A, m(k) + 2{ E^ {E{ m(k+1) ' (k+l)x(k+l) | x(k),m(k)}} 
-E T {d m(k) '(k)x(k)}}\x(k),m(k)} 



= I E ( % { I "<* W- D ^(k) (k) E m( k)( ^ (k+l),k) || wj (t) ! 

Jc=0 

+E{ | Cid,m(N)(N)x(N)+Di3d /m (N)(N)r d (N) | 2 } 

fc=0 



where 



ii d (k)= Ud(k)-F 2 , m (k)(k) x(k)-D u ,m(k)(k)r d (k). 
Since the left hand side reduces to 



N-l 

I 

A-0 



£ E{ E^ { | z d (k) | z }}+E{ | Cid,m(N)(N)x(N)+D 13 d,m(N)(N)rd(N) | 

+ E{20 m(N) ' (N)x(N)+x'(N)X m(N) (N)x(N)+« m(N) (N)} 
+E ( E i^ I" 2 3„ '(°) X (°K(°) X !o (°) x (°)- ««„ (0)}} 



noticing that the equality 



E^ {E{x'(N)X m(N) (N)x(N)+ a m(N) (N)+2 (9 n(N) '(N)x(N) | x(l),m(l)}} 
- E^ {x'(l)X m(1) x(l)+ « m(;) (l)+2 m(;) '(l)x(l)} 

N-l 

= I E f %^ {E{x'(k + l)X m(k+ i)(k + l)x(k + l)+ a m(k+1) (k+1) 
+ 2^ (M) '(k+l)x(k + l)|x(k) / m(k)}} 
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- E^ {x'(k)X m(k) x(k)+ a m(k) (k)+2 e m(k) '(k)x(k)} | x(l),m(l)} 

N-l 

+2{ E^ {E{ m(k+l) ' (k+l)x(k+l) | x(k),m(k)}} 
-{E^ mW '(k)x(k)}}x(l),m(l)} 

holds for 1, < 1 < N-l, we obtain 

JdN(xo,Ud,r d )=tr{ Q, o X, o }+ % (0)+E{ E^ {2 ^ '(0)x„}} 

N-l 

+E{ X E^ { | u d (k)-D eu , m(k) (k)E m(k) ( (k+l),k) || (t) }}+E{ /, (r d )} 



where we have used the terminal conditions X i (N)=C 1 d,i l (N)C ld ,i(N) / l (N)=C ld /Di 3d ,ir d (N) 
and a i (N)=0. Note that ] d (r d ) is independent of u d and x . Since the average of ff^mik) s ) 
over Rk is zero, including the 'causal' part Cim i k \ (k) of 6 (•) at time k, we adopt 



u d (k)= De u , m( k) (k) E m(k) ( 6 C (k+l),k) 



as the minimizing control strategy. 
Then finally we obtain 



J d N(xo,u d ,r d )=tr{ Q, o X, o }+ % (0)+E{ E^ {2 ^ '(0)x }} 

N-l 

+E{ X % (I fi„ (k)-D eu , m(k) (k)E m(k) ( (k+l),k) || a (t) }}+E{ ] d ( rd )} 

(r=0 

>tr{Q, o X !o } + «, o (O)+E{E^{2 0, o '(O)xo}} 
+ E{ X E^ { | D eu , m(k) (k)E m(k) ( ^ (k+l),k) ||^ )(t) }}+E{ J, (r d )} 



N-l 

z 

k-0 

~ J d N(XQ, Ujjd) 



which concludes the proof of sufficiency. 

Necessity: 

Because of arbitrariness of the reference signal r d (), by considering the case of r d () = 0, one 

can easily deduce the necessity for the solvability of the stochastic LQ optimal tracking 

problem [Costa et al. (2005); Fragoso (1989)]. Also notice that, in the proof of sufficiency, on 

the process of the evaluation of the performance index, by getting rid of the mixed terms of 
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rd and x, or # m m and x, we necessarily obtain the form of the preview compensator 
dynamics. (Q.E.D.) 

Appendix 2. Proof of Orthogonal Property (1 0) 

In this appendix we give the proof of the orthogonal property (10). 

We prove it by induction on k. 

For k=0, since x e (0) is deterministic, 



E{ E Ta {e(0) x e l (0)l {m( o ) .i ) }}= ff,-(0) E{ E^ {e(0)}} x e '(0)=O. 
We have already shown that, for each k e [0,N], 

E{E^{e(k)r rf , c 1 (k)l {m(k)=l| }}=0 

in section 4. Suppose 

E{E^{e(k)x e '(k)l {m(k) . i) }}=0. 

Then, since ffld(k) is zero mean, not correlated with x e (k) and n, c (k) and independent of 
m(k), we have 

E{E^{e(k + l)i/(k + l)l {m(k+lH }} 

= I Pd,;[[A d ,i +M i C i ](k)E(E s -{e(k)i e '(k)l (m(k)=i) }}[A d>i +M i C 2d , i ]'(k) 

WW 

+ [G d4 +M i H d , i ](k)E{E^{« d (k)x e '(k)l {m(kH }}[A cU + M i C 2d ,i]'(k) 

+ [A d4+ M I C 2 d,i]( k ) E { E ^{ e ( k )^* l (k)l |m(kH }}B 2d /(k) 

+ [G d , I+ M i H d4 ](k)E{E ¥ { Wd (k)^ 1 (k)l |m(kH }}B 2d , I '(k) 

+ [A d4+ M i C 2 d, i ](k)E{E^{e(k)r d , c '(k)l {m(kH }}B 3 d/(k) 

+ [G d4+ M i H tU ](k)E{E I -{« d (k)r rf /(k)l {m(kH }}B 3d /(k) 

-[A d4+ M i C 2d , i ](k)E{E I -{e(k)y'(k)l |m(kH }}M i '(k) 

-[G dii+ M i H d/i ](k)E{E^{« d (k)y'(k)l {m(kH }}M i '(k)] 

= I P^f"-[ A d, I +M i C 2 d,i]( k ) E { E ^{ e ( k )y l ( k )l |m (kH) i } M i'( k ) 
-[G d/i+ M i H d , i ](k)E{E^{« d (k)y'(k)l (m(k)=i) }}M i '(k)l 
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where ua, c * (k)=F 2/i (k) x e (k) , i=l, -,M. Notice that 

y(k)= C2d,m(k)(k)x(k)+H d ,m(k)(k)(Dd(k)= C 2d , m (k)(k)(e(k)+ x e (k))+H d , m (k)(k)cDd(k). 
Then, by induction on k, we obtain 

E{ E^ {e(k)y'(k)l {m(k)=i| }}= E{ E^ {e(k)e l (k)l im(k)=l| }}C 2d , 1 l (k)+E{ E^ {e(k) x e '(k)l tMm} }}C 2 ^(k) 

+ E{E^{e(k)a,d , (k)l |m( k ) =i l }}H d , 1 l (k) 
=Yi(k)C 2 d,i'(k) 
We also obtain 

E{ E^- {(Dd(k)y'(k)l |m (k)=i}}} 

= E{ E^ {cD d (k)e l (k)l {m (k)=i|}}C 2 d,i l (k)+E{ E^ jco d (k) x e '(k)l tm{m] }}C 2 d,'(k) 

+ E{ E^ {cD d (k)cfld l (k)l {m(k) = l| }}Hd,i l (k) 
= E{Q) d (k)cDd l (k)}P{m(k)=i}Hd, 1 l (k)= ni(k)H d ,i'(k). 

Then considering the assumption A4 G d/ i(k)H d ,i'(k) = O, i=l, -,M, and 

Mi(k)(H d , i Hd, 1 'n i (k)+ C 2 d,iYi(k)C 2 d,i')= - A d , i Y 1 (k)C 2d , 1 ' 
by (11), we finally obtain 

E{E J ^{e(k+l)x e 1 (k+l)l im(k+1) =i l }} 

* it+1 

= Z Pd,,j [-[Ad, 1 +M i C 2 d, 1 ](k)Y i (k)C 2d , 1 l (k)-[Gd,i+M i Hd,i](k)n i (k)Hd,i l (k)]M i '(k) 

,ej(k) 

= Z Pd,,j [-Ad,Y i (k)C 2d , 1 l (k)-M i (k)(H d , i Hd,i'n i (k)+ C 2d ,iY 1 (k)C 2 d,i')]M i '(k) 

iej(k) 

= £ Va.ij [-Ad,Y i (k)C 2d , 1 '(k)+ Ad,Yi(k)C 2d /]M i '(k) 

WW 

=0 

which concludes the proof. (Q.E.D.) 

7. References 

E. K. Boukas. (2006). Stochastic Switching Systems: Analysis and Design, Birkhauser, 0-8176- 

3782-6, Boston, 
A. Cohen. & U. Shaked. (1997). Linear Discrete-Time Hco-Optimal Tracking with Preview. 

IEEE Trans. Automat. Contr., 42, 2, 270-276 
O. L. V. Costa. & E. F. Tuesta. (2003). Finite Horizon Quadratic Optimal Control and a 

Separation Principle for Markovian Jump Linear Systems. IEEE Trans. Automat. 

Contr., 48, 10, 1836-1842 



Stochastic Optimal Tracking with Preview for Linear Discrete Time Markovian Jump Systems 129 

O. L. V. Costa.; M. D. Fragoso. & R. P. Marques. (2005). Discrete-Time Markov Jump Linear 

Systems, Springer, 1-85233-761-3, London 
V. Dragan. & T. Morozan. (2004). The linear quadratic optimization problems for a class of 

linear stochastic systems with multiplicative white noise and Markovian jumping. 

7EEE Trans. Automat. Contr., 49, 5, 665-675 
M. D. Fragoso. (1989). Discrete-Time Jump LQG Problem. Int. J. Systems Science, 20, 12, 2539- 

2545 
M. D. Fragoso.; J. B. R. do Val . & D. L. Pinto Junior. (1995). Jump Linear Hoc Control: the 

discrete-time case. Control-Theory and Advanced Technology, 10, 4, 1459-1474 
E. Gershon.; D. J. N. Limebeer.; U. Shaked. & I. Yaesh. (2004). Stochastic Hco Tracking with 

Preview for State-Multiplicative Systems. IEEE Trans. Automat. Contr., 49, 11, 2061- 

2068 
E. Gershon.; U. Shaked. & I. Yaesh. (2004). Hoo tracking of linear continuous-time systems 

with stochastic uncertainties and preview. Int. J. Robust and Nonlinear Control, 14, 7, 

607-626 
E. Gershon.; U. Shaked. & I. Yaesh. (2005). Hco Control and Estimation of State-Multiplicative 

Linear Systems, LNCIS 318, Springer, 1-85233-997-7, London 
J.-W. Lee. & P. P. Khargonekar. (2008). Optimal output regulation for discrete-time switched 

and Markovian jump linear systems, SIAM J. Control Optim., 47, 1, 40-72 
M. Mariton. (1990). Jump Linear Systems in Automatic Control, Marcel Dekker, 0-8247-8200-3, 

New York 
G. Nakura. (2008a). Noncausal Optimal Tracking for Linear Switched Systems. In: Hybrid 

Systems: Computation and Control: 11th International Workshop, HSCC 2008, St. Louis, 

MO, USA, April, 2008, Proceedings, LNCS 4981, M. Egerstedt. & B. Mishra. (eds.), 

pp.372-385, Springer, 3-540-78928-6, Berlin, Heidelberg. 
G. Nakura. (2008b). Hco Tracking with Preview for Linear Systems with Impulsive Effects - 

State Feedback and Full Information Cases-. Proceedings of the 17th IF AC World 

Congress, TuA08.4 (CD-ROM), Seoul, Korea 
G. Nakura. (2008c). Hco Tracking with Preview by Output Feedback for Linear Systems with 

Impulsive Effects. Proceedings of the 17th IF AC World Congress, TuA08.5 (CD-ROM), 

Seoul, Korea 
G. Nakura. (2008d). Stochastic Optimal Tracking with Preview for Linear Continuous-Time 

Markovian Jump Systems. Proceedings of SICE Annual Conference 2008, 2A09-2 (CD- 
ROM), Chofu, Tokyo, Japan 
G. Nakura. (2008e). Hco Tracking with Preview for Linear Continuous-Time Markovian 

Jump Systems. Proceedings of SICE 8th Annual Conference on Control Systems, 073-2-1 

(CD-ROM), Kyoto, Japan 
G. Nakura. (2009). Stochastic Optimal Tracking with Preview for Linear Discrete-Time 

Markovian Jump Systems (Extended Abstract). In: Hybrid Systems: Computation and 

Control: 12th Conference, HSCC 2009, San Francisco, CA, USA, April, 2009, Proceedings, 

LNCS 5469, R. Majumdar. & P. Tabuada. (Eds.), pp. 455-459, Springer, 3-642-00601- 

9, Berlin, Heidelberg 



130 Discrete Time Systems 

G. Nakura. (2010). Stochastic Optimal Tracking with Preview by State Feedback for Linear 

Discrete-Time Markovian Jump Systems. International Journal of Innovative 

Computing, Information and Control (YjICIC), 6, 1, 15-27 
Y. Sawada. (2008). Risk-sensitive tracking control of stochastic systems with preview action. 

International Journal of Innovative Computing, Information and Control (IJICIC), 4, 1, 

189-198 
U. Shaked. & C. E. de Souza. (1995). Continuous-Time Tracking Problems in an Hco Setting: 

A Game Theory Approach. IEEE Trans. Automat. Contr., 40, 5, 841-852 

C. E. de Souza. & M. D. Fragoso. (1993). Hco Control for Linear Systems with Markovian 

Jumping Parameters. Control-Theory and Advanced Technology, 9, 2, 457-466 

D. D. Sworder. (1969). Feedback Control of a Class of Linear Systems with Jump Parameters. 

IEEE Trans. Automat. Contr., AC-14, 1, 9-14 
D. D. Sworder. (1972). Control of Jump Parameter Systems with Discontinuous State 

Trajectories. IEEE Trans. Automat. Contr., AC-17, 5, 740-741 
K. Takaba. (2000). Robust servomechanism with preview action for polytopic uncertain 

systems. Int. J. Robust and Nonlinear Control, 10, 2, 101-111 



8 



The Design of a Discrete Time Model Following 
Control System for Nonlinear Descriptor System 

Shigenori Okubo 1 and Shujing Wu 2 

l Yamagata University 

2 Shanghai University of Engineering Science 

1 }apan 
2 P. R. China 

1. Introduction 

This paper studies the design of a model following control system (MFCS) for nonlinear 
descriptor system in discrete time. In previous studies, a method of nonlinear model 
following control system with disturbances was proposed by Okubo,S. and also a nonlinear 
model following control system with unstable zero of the linear part, a nonlinear model 
following control system with containing inputs in nonlinear parts, and a nonlinear model 
following control system using stable zero assignment. In this paper, the method of MFCS 
will be extended to descriptor system in discrete time, and the effectiveness of the method 
will be verified by numerical simulation. 

2. Expressions of the problem 

The controlled object is described below, which is a nonlinear descriptor system in discrete 
time. 

Ex(k + 1) = Ax(k) + Bu(k) + B f f(v(k)) + d(k) (1) 

v(k) = C f x(k) (2) 

y{k) = Cx{k) + d Q {k) (3) 

The reference model is given below, which is assumed controllable and observable. 

x m (k + l) = A m x m (k) + B m r m (k) (4) 

y,n( k ) = C „, X n,( k ) ( 5 ) 

, where 

x(k) e R",d(k) e R n ,u(k) e R l ,y(k) e R l ,yjk) e R ' ,d (k) e R e ,f(v(k)) e R 1 ' , 

v(k) e R ' ,r m (k) eR ••• ,x m (k) s R" ,n , y(k) is the available states output vector, v(k) is the 
measurement output vector, u(k) is the control input vector, x(k) is the internal state vector 
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whose elements are available, d(k),d (k) are bounded disturbances, y m {k) is the model 

output. 

The basic assumptions are as follows: 

1. Assume that (C,A,B) is controllable and observable, i.e. 



rank[zE - A,B] = n,rank 



zE-A 
C 



2. In order to guarantee the existence and uniqueness of the solution and have exponential 
function mode but an impulse one for (1), the following conditions are assumed. 

|zE-A|#0, rankE = deg|zE- A\ = r <n 

3. Zeros of C[zE-A] B are stable. 

In this system, the nonlinear function f(v(k)) is available and satisfies the following 
constraint. 

\\f{v{k))\<a + p\v{k)f, 

where a >0,/7 >0,0 < y <1, INI is Euclidean norm, disturbances d(k),d (k) are bounded and 
satisfy 

D d (z)d(k) = (6) 

D d (z)d (k) = 0. (7) 

Here, D d (z) is a scalar characteristic polynomial of disturbances. Output error is given as 

e(k) = y(k)-yjk). (8) 

The aim of the control system design is to obtain a control law which makes the output error 
zero and keeps the internal states be bounded. 

3. Design of a nonlinear model following control system 

Let z be the shift operator, Eq.(l) can be rewritten as follows. 

C[zE-Ay 1 B = N(z)/D(z) 

C[zE-Ay 1 B f =N f (z)/D(z), 

where D(z) = \zE-A\, d Ij (N(z)) = a l and d r ,(N f (z)) = a fi . 
Then the representations of input-output equation is given as 

D(z)y(k) = N(z)u(k) + N f (z)f(v(k)) + w(k) . (9) 

Here w(k) =Cadj[zE- A]d(k) + D(z)d (k) , (C m ,A m ,B m ) is controllable and observable. Hence, 

C m [zI-A m ]- 1 B m =N m (z)/D m (z). 
Then, we have 
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D m (z) y , n (k)=N m (z)r m (k), (10) 

where D m (z) = \zl - A m \ and d r .(N m (z)) = a„, . . 

Since the disturbances satisfy Eq.(6) and Eq.(7), and D d (z) is a monic polynomial, one has 

D d (z)w(k)=0. (11) 

The first step of design is that a monic and stable polynomial T(z) , which has the degree of 
p(/?>n d +2n-n m -1 — CTj), is chosen. Then, R(z) and S(z) can be obtained from 

T(z)D m (z) = D d (z)D(z)R(z) + S(z), (12) 

where the degree of each polynomial is: dT(z) = p,dD d (z) = n d ,dD m (z) = n m , 
8D(z) = n,dR(z) = p + n m -n d -n and 8S(z) <n d +n -1 . 
From Eq.(8)~(12), the following form is obtained: 

T(z)D m (z)e(k) = D d (z)R(z)N(z)u(k) + D d (z)R(z)N f (z)f(v(k)) 
+ S(z)y(k)-T(z)N m (z)rJz). 
The output error e(k) is represented as following. 

e ^ = T , * ; - {[D d (z)R(z)N(z) - Q(z)N r ]u(k) + Q(z)N r u(k) n „ 

T (z)D m (z) (13) 

+ D d (z)R(z)N f (z)f(v(k)) + S(z)y(fc) - T(z)N ffl (z)r m (fc)} 

Suppose r r (N(z)) = N r , where r r (-) is the coefficient matrix of the element with maximum 
of row degree, as well as \N r \ ^ . The next control law u(k) can be obtained by making the 
right-hand side of Eq.(13) be equal to zero. Thus, 

u(k) = -N; 1 Q- 1 (z){D d (z)R(z)N(z)-Q(z)N r }u(k) (14) 

-N; 1 Q-\z)D d (z)R(z)N f (z)f(v(k))-N; 1 Q- 1 (z)S(z)y(k) + u m (k) 

u m (k) = N; 1 Q- 1 (z)T(z)N m (z)r m (k) . (15) 

Here, Q(z) = diagl z S ' \,8 i = p + n m -n + cr i (i = l, 2, ■■■,£), and u(k) of Eq. (14) is obtained from 
e(k) = 0. The model following control system can be realized if the system internal states are 
bounded. 

4. Proof of the bounded property of internal states 

System inputs are both reference input signal r m {k) and disturbances d(k),d (k), which are 
all assumed to be bounded. The bounded property can be easily proved if there is no 
nonlinear part f(v(k)) . But if f(v(k)) exits, the bound has a relation with it. 
The state space expression of u(k) is 

«(*) = -H&(k) - E 2 y(k) - H 2 { 2 (k) - E 3 f(v(k)) - H 3 &(*) + ujk) (16) 

u m (k) = E A r m (k) + H^ 4 (k). (17) 
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The following must be satisfied: 



% 1 (k + l) = F 1 % 1 (k) + G 1 u(k) 

4 2 (k + l) = F 2 ^(k) + G 2 y(k) 

t 3 (k + l)=F 3 { 3 (k) + G 3 f(v(k)) 

^(k + l) = F i ^(k) + G 4 r m (k). 



(18) 
(19) 
(20) 
(21) 



Here, 

|zT-F,| = |Q(z)|, (t = l,2,3,4). 

Note that there are connections between the polynomial matrices and the system matrices, 

as follows: 



N r - 1 Q- 1 (2){D,( 2 )_R(2)N( 2 )-Q( 2 )N r }=H 1 (2l-F 1 )- 1 G 1 



N; 1 Q-\z)S(z) = H 2 (zl - F 2 y 1 G 1 + E 2 



K 1 Q- 1 (z)DAz)R(zW f (z) = H 3 (zi-F 3 y 1 G 3 + E 3 



N; 1 Q- 1 (z)T(z)N m (z) = H i (zI-F i )- 1 G i+ E i . 



(22) 
(23) 
(24) 
(25) 



Firstly, remove u(k) from Eq.(l)~(3) and Eq.(18)~(21). Then, the representation of the 
overall system can be obtained as follows. 



E 





0" 


' x(k + l) 





I 





6(* + i) 








I 


&(* + i) 








I 


U( fc+i )J 



A-BE 2 C -BH-! 


-BH 2 


—G^E 2 C Fj— GjHj 


_G 1 H 2 


G 2 C 


F 2 









-BH 3 ~| 


x(k) 


G 1 H 3 


m) 





&(*) 


F3 


Imu 



(26) 





~BH A ' 




Bjr - BE 3 




"BE 4 " 




+ 


G 1 H 4 





m)+ 


-G a E 3 

G 3 


f(v(k)) + 


G a E 4 




r m (k) + 



d{k)-BE 2 d {k) 


-G 1 E 2 d (fc) 


G 2 d (fc) 






£ 4 (fc + l) = F 4 | 4 (lc) + G 4 r„,(lc) 



(27) 
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p(fc) = [C / o] 



feW 

few 
few 



(28) 



y(*) = [C 0] 



feW 



+d (fc). 



(29) 



In Eq. (27), the £ 4 (fc) is bounded because Izl-Fj =|Q(z)| is a stable polynomial and r m (k) is 
the bounded reference input. Let z(k),A s ,E,d s (k),B s ,C v ,C s be as follows respectively: 



z(k) = [x T (k) g(k) ? 2 {k) g(k)J, As 



-BU-i 



-BH-, 



-BH 3 



-G^£2^- t^—G-Jri^ —G^ri2 ~ G-f-fig 



o 



E 














I 














I 














I 



<*,(*) = 



Bu m (k) + d(k)-BE 2 d (k) 

G lMm (fc)-G 1 E 2 d (fc) 

G 2 d (k) 







Bf - BE 3 


/ B s = 


-G a E 3 





G 3 



c v =\c f o o], c s =[c 0], 



With the consideration that ^(k) is bounded, the necessary parts to an easy proof of the 
bounded property are arranged as 



Ez(k + l) = A s z(k) + BJ(v(k)) + d s (k) 

v(k) = C v z(k) 
y(k) = C s z(k) + d (k), 



(30) 
(31) 
(32) 



where the contents of A s ,E,d s (k),B s ,C v ,C s are constant matrices, and d s (k) is bounded. 
Thus, the internal states are bounded if z(k) can be proved to be bounded. So it needs to 
prove that \zE - A s \ is a stable polynomial. The characteristic polynomial of A s is calculated 
as the next equation. 
From Eq.(26), \zE - A s \ can be shown as 



\zE-A. \ = 



BH t 



BH-, 



BH, 



-G 2 C 




zI-F 1 +G 1 H 1 G t H 2 G t H 3 



zI-F, 




zI-F, 



(33) 
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Prepare the following formulas: 



X Y 
W Z 



izllx-yz-HvLdzl^o), 



i-x(i + yx) _1 y = (i + xyy 1 

\I + XY\ = \I + YX\. 
Using the above formulas, zE-A s is described as 

| Z E"4| 
= \zI-F 3 \\zI-F 2 \\zI-F 1 \\l + H 1 [zI-F 1 Y 1 G 1 \ 

■ \zE-A + B{I-H 1 [zI-F 1 +G 1 H 1 Y 1 G 1 }{E 2 +H 2 [zI-F 2 y 1 G 2 }c\ 

= \Q(zf\l + H 1 [zI-F 1 Y 1 G 1 \\zE-A + B{I + H 1 [zI-F 1 Y 1 G i y 1 {E 2 +H 2 [zI-F 2 y 1 G 2 }c\ 

= |Q(z)| 3 |/ 1 || Z E-A||l + B/ 1 - 1 / 2 [zE-A]- 1 | 

= |Q(z)| 3 | Z E-A||/ 1 +/ 2 [zE-Af 1 B|. 

Here 

] 1 =I + H 1 [zI-F i y 1 G 1 

] 2 ={E 2 +H 2 [zI-F 2 ]- 1 G 2 }C. 
From Eq.(22),(23),(35) and Eq.(36), we have 

h=N; 1 Q- 1 (z)D d (z)R(z)N(z) 

] 2 =N; 1 Q- 1 (z)S(z)C. 
Using C[zE-A] B = N(z) / D(z) and D(z) = |zE-A|, furthermore, zE-A s is shown as 



(34) 



(35) 
(36) 

(37) 
(38) 



\zE-A s \ = T l (z)D t m (z)\Q(z) 



: |N(2)||N r f 
D'-Hz) 



and V(z) is the zeros polynomial of C[zE-A] B = N(z) / D(z) = U 1 (z)V(z) (left coprime 
decomposition), |lZ(z)| = D(z) , that is, \N(z)\ = D e ~ 1 (z)\V(z)\ . So zE-A s can be rewritten as 



\zE-A s \ = \N r [ 1 T e (z)D t m (z)\Q(zf\V(z)\ 

As T(z),D m (z),|Q(z)|,|V(z)| are all stable polynomials, A s is a stable system matrix. 
Consider the following: 



(39) 
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%{k) 

Using Eq.(40), one obtains 

PEQz(k + 1) = PA s Qz(k) + PB,f(p(k)) + Pd s (k) . 



Namely, 





I 


z(k + l) 




Ai o 


z\(k) 







z 2 (k + l) 




I 


[z 2 (k) 


One can rewrittei 


i Eq.(4 


l)as 









B„ 



f(v(k))- 



d sl (k) 
d s2 (k) 



z 1 {k + l) = A sl z 1 {k) + B sl f{v{k)) + d sl {k) 

= z 2 (k) + B s2 f(v(k)) + d s2 (k) 
. where z(k),Pd s (k),PA s Q,PB s can be represented by 



z(k) = 


_z 2 (k)\' sK > 


sU ; , PAn = 


'An °~ 
I 


, PB S = 


_ B S 2_ 


LetC„Q = [C rl C, 2 ],then 




v(k) = C vl z 1 (k) + C v2 z 2 (k). 




From Eq.(43) and Eq.(45), we have 




v(k) + C v2 B s2 f(v(k)) = C vl z(k) - C v2 d s2 (k) . 




From Eq.(46), we have 




-^—Mk) + C v2 B s2 f(v(k)) = I + C v2 B s2 §^ . 
dv (k) dv (k) 




Existing condition of 


v(k) is 











I + C 



df(v(k)) 



dv (k) 



*0. 



From Eq.(44), we have 

|P||zE-A s ||Q| = « PQ |zE-A s 



zI-A sl 
-I 



■ aAzI — AgJ 



Here, oc P q and a l are fixed. So, from Eq.(39), A sl is a stable system matrix. 
Consider a quadratic Lyapunov function candidate 



(40) 



(41) 

(42) 
(43) 

(44) 

(45) 
(46) 



(47) 



(48) 



V{k) = zl{k)P s \{k). 



(49) 
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The difference of V(k) along the trajectories of system Eq.(42) is given by 

AV{k) = V(k + l)-V(k) 

= z[{k + l)P s z a (fc + 1) - z\{k)P s z x {k) (50) 

= [Ai&Qc) + B sl f(v(k)) + d sl (k)f P s [A sl \(k) + B sl f(v(k)) + d sl {k)}-zl{k)P s \^) 

A T A P s A sl -P s =-Q s , (51) 

where Q s and P s are symmetric positive definite matrices defined by Eq.(51). If A sl is a 
stable matrix, we can get a unique P s from Eq.(51) when Q s is given. As d sl (k) is bounded 
and < / < 1 , AV(k) satisfies 



(52) 



AV(k) < -zf(k)Q s z t (k) + X, 1^(^)111/(^^)1 

+ X 2 |z a (fc)| + A2 \\f(v(k))f + x 3 l/^fc))! + x 4 

From Eq.(40), we have 

||z a (fc)| <M\\z(k)\\. (53) 

Here, M is positive constant. From Eq.(52), Eq.(53), we have 



AV(k)<- Ml \\z(k)f + X 5 \z l (k)f +r + X 6 

<- Mc \\z(k)f+X 

<- Mm V(k) + X, 



(54) 



where < // a = /L min (Q s ),,« 2 >0 and < ju m < ju c <min(/i 1 ,l) . Also, // 1/( u 2 , X ! (i=l~6) and 
X are positive constants. As a result of Eq.(54), V(fc) is bounded: 

V(k)<V(0) + X/ Mm . (55) 

Hence, Zj(fc) is bounded. From Eq.(43), z 2 (A:) i s a l so bounded. Therefore, z(k) is bounded. 

The above result is summarized as Theoreml. 

[Theoreml] 

In the nonlinear system 

Ex(k + 1) = Ax(k) + Bu(k) + B f f(v(k)) + d{k) 

v(k) = C f x(k) (56) 

y(k) = Cx(k) + d (k), 

where x{k)eR" ,u(k) eR ',y(k)eR ' ,v(k) eR 1 ' ,d(k) eR" ,d (k) eR 1 ' ,f(v(k))eR f ', d(k) and 
d (k) are assumed to be bounded. All the internal states are bounded and the output error 
e(k) = y(k)-y m (k) asymptotically converges to zero in the design of the model following 
control system for a nonlinear descriptor system in discrete time, if the following conditions 
are held: 
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1. Both the controlled object and the reference model are controllable and observable. 

2. |N r |*0. 

3. Zeros of CfzE-Al : B are stable. 

4. \\f(v(k))\\ < a + 0\\v(k)( ,(a > 0,fi > 0,0 < y < 1) . 

8f(v(k)) 



I + C. 



5. Existing condition of v(k) is ± v ,u, 

6. |zE-A|#0 and rankE = deg|zE- A\ = r <n . 



*0- 



5. Numerical simulation 

An example is given as follows: 



"1 1 




Oil 


x(fc + l) = 


10 1 








1 " 




"0 0" 




1 




"0" 





1 


*(*) + 


1 


u(k) + 





/(»(*))+ 


1 


0.2 


-0.5 0.6 




1 




1 




1 



i(fc) 



»(fc) = [l 1 l]x(fc), 
"0 0.1 " 
0.1 0.1 



m- 



*(*)+ 



/(p(fc)) 



3u 3 (fc) + 4i;(fc) + l 
l + u 4 (/c) 



Reference model is given by 

X m (k + 1) 



1 
-0.12 0.7 

y»(*)=[i o]* m (k) 

r m (k)=sin(te/16). 



X m( fc )- 



(57) 



r„(fc) 



(58) 



In this example, disturbances d(A:) and d (A:) are ramp and step disturbances respectively. 
Then d(k) and d (k) are given as 



rf(fc) = 0.05(fc - 85),(85 < k < 100) 
d (k) = 1.2,(20 <fc<50) 



(59) 



We show a result of simulation in Fig. 1. It can be concluded that the output signal follows 
the reference even if disturbances exit in the system. 



6. Conclusion 

In the responses (Fig. 1) of the discrete time model following control system for nonlinear 
descriptor system, the output signal follows the references even though disturbances exit in 
the system. The effectiveness of this method has thus been verified. The future topic is that 
the case of nonlinear system for y>\ will be proved and analysed. 
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Fig. 1. Responses of the system for nonlinear descriptor system in discrete time 
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1. Introduction 

Most physical systems have only limited states to be measured and fed back for system 
controls. Although sometimes, a reduced-order observer can be designed to meet the 
requirements of full-state feedback, it does introduce extra dynamics, which increases the 
complexity of the design. This naturally motivates the employment of output feedback, which 
only use measurable output in its feedback design. From implementation point of view, static 
feedback is more cost effective, more reliable and easier to implement than dynamic feedback 
(Khalil, 2002; Kucera & Souza, 1995; Syrmos et al., 1997). Moreover, many other problems are 
reducible to some variation of it. Simply stated, the static output feedback problem is to find 
a static output feedback so that the closed-loop system has some desirable characteristics, or 
determine the nonexistence of such a feedback (Syrmos et al., 1997). This problem, however, 
still marked as one important open question even for LTI systems in control engineering. 
Although this problem is also known NP-hard (Syrmos et al., 1997), the curious fact to 
note here is that these early negative results have not prevented researchers from studying 
output feedback problems. In fact, there are a lot of existing works addressing this problem 
using different approaches, say, for example, Riccati equation approach, rank-constrained 
conditions, approach based on structural properties, bilinear matrix inequality (BMI) 
approaches and min-max optimization techniques (e.g., Bara & Boutayeb (2005; 2006); Benton 
(Jr.); Gadewadikar et al. (2006); Geromel, de Oliveira & Hsu (1998); Geromel et al. (1996); 
Ghaoui et al. (2001); Henrion et al. (2005); Kucera & Souza (1995); Syrmos et al. (1997) and the 
references therein). Nevertheless, the LMI approaches for this problem remain popular (Bara 
& Boutayeb, 2005; 2006; Cao & Sun, 1998; Geromel, de Oliveira & Hsu, 1998; Geromel et al., 
1996; Prempain & Postlethwaite, 2001; Yu, 2004; Zecevic & Siljak, 2004) due to simplicity and 
efficiency. 

Motivated by the recent work (Bara & Boutayeb, 2005; 2006; Geromel et al., 1996; Xu & Xie, 
2005a;b; 2006), this paper proposes several scaling linear matrix inequality (LMI) approaches 
to static output feedback control of discrete-time linear time invariant (LTI) plants. Based on 
whether a similarity matrix transformation is applied, we divide these approaches into two 
parts. Some approaches with similarity transformation are concerned with the dimension 
and rank of system input and output. Several different methods with respect to the system 
state dimension, output dimension and input dimension are given based on whether the 
distribution matrix of input B or the distribution matrix of output C is full-rank. The other 
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approaches apply Finsler's Lemma to deal with the Lyapunov matrix and controller gain 
directly without similarity transformation. Compared with the BMI approach (e.g., Henrion 
et al. (2005)) or VK-like iterative approach (e.g.,Yu (2004)), the scaling LMI approaches are 
much more efficient and convergence properties are generally guaranteed. Meanwhile, they 
can significantly reduce the conservatism of non-scaling method, (e.g.,Bara & Boutayeb (2005; 
2006)). Hence, we show that our approaches actually can be treated as alternative and 
complemental methods for existing works. 

The remainder of this paper is organized as follows. In Section 2, we state the system and 
problem. In Section 3, several approaches based on similarity transformation are given. In 
Subection 3.1, we present the methods for the case that B is full column rank. Based on 
the relationship between the system state dimension and input dimension, we discuss it in 
three parts. In Subsection 3.2, we consider the case that C is full row rank in the similar way. 
In Subsection 3.3, we propose another formulations based on the connection between state 
feedback and output feedback. In Section 4, we present the methods based on Finsler 's lemma. 
In Section 5, we compare our methods with some existing works and give a brief statistical 
analysis. In Section 6, we extend the latter result to H<x> control. Finally, a conclusion is given 
in the last section. The notation in this paper is standard. TV 1 denotes the n dimensional real 
space. Matrix A > (A > 0) means A is positive definite (semi-definite). 

2. Problem formulation 

Consider the following discrete- time linear time-invariant (LTI) system: 

x(t + l) = A x{t) + B u{t) (1) 

y(t) = Cax(t) (2) 

where x 6 TV 1 , U 6 TV and y € 72- . All the matrices mentioned in this paper are appropriately 

dimensioned, in < n and I < n. 

We want to stabilize the system (l)-(2) by static output feedback 

u(t) = Ky(t) (3) 

The closed-loop system is 

x(t + 1) = Ax{t) = (A + B KC )x{t) (4) 

The following lemma is well-known. 

Lemma 1. (Boyd et al, 1994) The closed-loop system (4) is (Schur) stable if and only if either one of 
the following conditions is satisfied: 

P > 0, A T PA - P < (5) 

Q>0, AQA T -Q<0 (6) 

3. Scaling LMIs with similarity transformation 

This section is motivated by the recent LMI formulation of output feedback control (Bara 
& Boutayeb, 2005; 2006; Geromel, de Souze & Skelton, 1998) and dilated LMI formulation 
(de Oliveira et al., 1999; Xu et al., 2004). 
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3.1 B with full column-rank 

We assume that B is of full column-rank, which means we can always find a non-singular 

In fact, using singular value decomposition (SVD), we can 



matrix T/, such that TyB 







obtain such T},. Hence the new state-space representation of this system is given by 



A = T b A T, 



A u A 12 
A 2 \ A 2 2 



Ti,B , C — C T, 



-l 



(7) 



The closed-loop system (4) is stable if and only if 

A b = A + BKC is stable 
In this case, we divide it into 3 situations: m = n — m, m < n — m, and m > n — m. Let 

~Kl Pl2~ 
12 



Pi P22 



e n nx ",P u e K mxm ,P 12 e ii mx ("-'") 



(8) 



For the third situation, let 



P12 = \P$ P&\ Pn 



p(l) p(2) 
Ml Ml 
p(2)T p(3) 

Ml Ml 



(9) 



where P$ £ T&-~ m ) x ("-'") and P^ G -fl(n-m)x(n-m.\ 

Theorem 1. The discrete-time system (l)-(2) is stabilized by (3) if there exist P > defined in (8) and 
R, such that 

!<t>(@i) < 0, m = n - m 
<E>(® 2 ) < 0, m < n - m (10) 

4>(03) < 0, m > n — m 



where e G TZ, 



«!>(©! 



e. 



A T 0!A-P * 
RC+[P u P 12 }A-P n 





< 





P22 + e 2 Mi - £M2 - £ M T 2 



2 



©3 





P 22 - e 









- e[PJ 2 0] + £ 2 



MlO 





QFn-tpW -&$ + &$ 



Furthermore, a static output feedback controller gain is given 



K = P-LR 



Proof: Noting that 



(BKQtpiBKC) = C^^nKC, 



(11) 
(12) 



(13) 
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PBKC 



m 

J 12 



KC 



(5) is equivalent to 



([Pu Pu]A + P n KC) T Pu(l p u Pn]A + P n KC) 
Pu [Pu Pu}A + A T PA - P < 



Pu 

P T 



P- 



P u [PnP 



12J 



Considering that 

"0 

_0P 22 -Pl 2 P u 1 P u 

For the first situation m = n — m, consider the following inequality: 

(Pi2-eP 11 ) T P{ l 1 (P 12 -£P n )>0 



Pu 
P T 

J 12 



or equivalently 
(14) is equivalent to 

where 



p i2 p u p i2 > ePu + £p u - z 2 Pu 



A T @ A -P * 

P xl KC+[P n P 12 ]A-P n 



<0 



(14) 



(15) 
(16) 
(17) 



©n 



P22 - Pi 2 Pu p n 



Using the fact (16), we have ©0 < ©1, and consequentially, 4>(©rj) < 4>(©i). Hence if (11) is 

satisfied, (5) is satisfied as well. 

For the second situation, let the inequality 



'Pu 



— e 


"Pu 0" 



r 


"Pu 0" 
I 


"( 


'Pu 



— £ 


"PuO" 




>0 



(18) 



where 



P12 




elz (n-m)x(n-m) and 



PuO 

I 



p Ii p u p i2 > £ 



For the third situation, noting that 



Pu 





€ TZ 1 -"-'^ x ("-'«). Note that (18) is equivalent to 

(19) 



+ £ 



Pu 





PuO 




([Pu 0]-eP n ) T P n 1 ([Pu 0] - eP u ) > 



we have 



(21) implies 



l \2 




Pu\PnO\ >e I 2 +e[P 12 0}-e 2 P 



P T P- 1 ?,, > F P (1)r 4- FP (1) - f 2 P (1) 

1 U 1 11 L 12 ^ £J 12 + el 12 £ 1 11 



(20) 
(21) 

(22) 
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Hence we complete the proof. 

Remark 1. Ife = is set , then Theorem 1 recovers the result stated in (Bara & Boutayeb, 2006). We 
shall note that e actually plays an important role in the scaling LMI formulation in Theorem l.Ife = 0, 
Theorem 1 implies A 22 P22^22 — Pn < and P22 > 0, i.e., the system matrix A22 must be Schur 
stable, which obviously is an unnecessary condition and limits the application of this LMI formulation. 
However, with the aid of e, we relax this constraint. A searching routine, such as fminsearch (simplex 
search method) in Matlab ®, can be applied to the following optimization problem (for a fixed e, we have 
an LMI problem): 

minAi, s.t. <£(©) < A7 (23) 

e,P,R 

The conservatism of Theorem I lies in these relaxations (15) or (16) on (5). To further relax the 
conservatism, we may choose a diagonal matrix A = diag{e%,. ..,£„,},£, > 0, instead of the single 
scalar e. For example, 

PuPn p u > p i T 2 A + Ap i2 - Ap n& (24) 

Then we shall search the optimal value over multiple scalarsfor (23). 

Remark 2. In (Bara & Boutayeb, 2006), a different variable replacement is given: 



„ 22 



PliPnPn 



(25) 



in (8). However, it is easily proved that these two transformations actually are equivalent. In fact, in 
(8), we have Pn > and P2 > since P > 0. Based on (17), we have 



A-A Q 





0P 2 
P n KC+{P n P 12 ]A-P u 



<0 



(26) 



where 



A 



Pn P12 

Pl 2 P2 + Pl 2 Pn p i2 



(27) 



Hence, for the above three situations, we have an alternative condition, which is stated in the 
following lemma. 

Theorem 2. The discrete-time system (l)-(2) is stabilized by (3) if there exist Pn > 0, P2 > 0, P\2 
and R with P defined in (27), such that 



Y(Ai) < 0, m = n - m 
Y(A 2 ) < 0, m < n - m 
Y(A 3 ) < 0, m > n - m 



(28) 



where £€ K, 



Y(A0 

Ai = 





A T 
RC 




0P 2 

"+ [Pi 


A - A, * 
Y P l2 \A -P n 


Pi 

n 


1 

2^2- 


- £ 2 Pl1 


P12 

+ eP u + eP? 2 
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A, 



"Pll 




Pu 






P? 2 P2 + e 


J J 12 




+ e[P£0] 


-e 2 


Pu 
I 



A 3 



Pll Pl2 

_p£ft+ e p£ )T + e p£ ) -e 2 p 1 ( 1 1) 

Furthermore, a static output controller gain is given by (13). 



Proof: We only consider the first case. Replacing P 2 and R by P22 and K using (25) and (13), we 
can derive that (28) is a sufficient condition for (5) with the P defined in (8). 



3.2 C with full row-rank 

When Co is full row rank, there exists a nonsingular matrix T c such that C Tg 
Applying a similarity transformation to the system (l)-(2), the closed-loop system 
if and only if 

A c = A + BKC is stable 

where A = T C A T- 1 , B = T C B and C = CoT" 1 = [I, 0]. 

Similarly to Section 3.1, we can also divide this problem into three situations: 

I < n — I and / > n — I. We use the condition (6) here and partition Q as Q = 

where Q n G TZ lyA . 

Theorem 3. The discrete-time system (l)-(2) is stabilized by (3) if there exist Q > and 

( r(©i) < 0, l = n - 1 
I r(© 2 ) <0,l<n-l 
{ r(© 3 ) < 0, l > n - I 

A©,A T - Q * 

(A[Q n Qu} T + BR) T -Q n 



1 = [h 0]. 
(4) is stable 



I = n-l 

Qn Q12 

,T 
12 



Qi T 2 Q22 



where e €TZ, 



r(0, 



R, such that 
(29) 



©! 















Q22 + £ 2 Qn - £Qi2 - £Qi T 2 



©2 



Q 22 + £ 2 



Q11O 





Q12 





- £[Qi T 2 o] 



©3 



0Q 2 2 + e 2 Q ( u ] -eQ[l )T -eQ\ 1) 



'-YI 



Q u ' and Q ( 12 ' are properly dimensioned partitions of Qn and Q 12 . Furthermore, a static output 



feedback controller gain is given by 



K = RQ 



11 



(30) 
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Proof: We only prove the first case / = n — I, since the others are similar. Noting that 
(BKC)Q{BKC) T = BKQ U K T B and BKCQ = BK[Q n Q 12 ], (6) is equivalent to 



(MQu Qu] T + BKQn)Qi?(A[Q n Q12F + BKQ n ) 
Q u 1 [QuQi2\A T + AQA T -Q<0 



Using the fact that 



-A 



Q 



Qu 



Qi T 



(31) 



Q11 

,T 
12 



Ql 



QulQnQu] 





Q{ 2 Q n l Qn 



we infer that stability of the close-loop system is equivalent to the existing of a Q > such 
that 

AG A T - Q * 

(A[QuQi2]+BKQ u ) T -Q n 

where 



<0 



(32) 



©n 



Q22 - Q{ 2 QuQi2 



Since 

or equivalently, 



(Qn - eQn) T Qu(Qi2 - eQn) > 



QliQnQn > eQi T 2 + £<2i2 - £ 2 Qn 



(33) 
(34) 



It follows that (29) implies (32). Hence we complete the proof. 



Remark 3. How to compare the conditions in Theorem 3 and Theorem 1 remains a difficult problem. 
In the next section, we only give some experiential results based on numerical simulations, which give 
some suggestions on the dependence of the results with respect to m and I. 

3.3 Transformation-dependent LMIs 

The result in this subsection builds a connection between the sets C, fC c , ICo, K. c and K, 0/ which 
are defined as follows. Without causing confusion, we omit the subscript for A , B and C 
in this subsection. 

C = {K e 1l mxl : A stable} (35) 

i.e., the set of all admissible output feedback matrix gains; 



K c = {K c e TZ mxn :A + BK C stable} 
i.e., the set of all admissible state feedback matrix gains; 

/C = {K G n n/l :A + K C stable} 



(36) 



(37) 



i.e., the set of all admissible observer matrix gains. Based on Lemma 1, we can easily formulate 
the LMI solution for sets K, c and K, - In fact, they are equivalent to following two sets 
respectively: 

>Cc = {K c = WaW' 1 e Tl mxn : (W cl , W c2 ) G W c } (38) 
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and 



where Y r 



w c = {w cl eTZ nxn ,w c2 eTZ" 
-w cl 



Wei > 0,Y C < 0} 

AW d + BW c2 " 
W cl A T + Wj 2 B T -W cl 

K.o = {Ko = W^W o2 G 1l nKl : (W ol ,W o2 ) G W } 



and 



(39) 

(40) 
(41) 



where Y 



W = {W ol G ft" x ",W o2 G 7l" x ' : W ol > 0,Y o < 0} 

-W l0 W ol A + W o2 C" 

A T W ol + C T Wj 2 -W l0 

Lemma 2. £ 7^ ifanii only if 

1. K, c = K. c f]{Kc ■ K C Y C = 0,Y C = Af(C)} £ 0; or 

2. K = K f]{Kc : Y K = 0, Y = Af(B')} ^ 0. 
In the affirmative case, any K G £ can be rewritten as 

1. K = K c QC T (CQC T )- 1 ;or 

2. K = (B T PB)- 1 B T PK . 

where Q > and P > are arbitrarily chosen. 

Proof: The first statement has been proved in Geromel et al. (1996). For complement, we 
give the proof of the second statement. The necessity is obvious since K = BK. Now we 
prove the sufficiency, i.e., given K G K, , there exists a K, such that the constraint K = BK 

~B T P~ 
Y T 

1 n 



is solvable. Note that for VP > 0, ©„ 



is full rank, where Y = JV(B T ). In fact, 



rank(@ Y ) = rank( 



B T PY 
In—m 



) > n — m. Multiplying © at the both side of K = BK we have 



B T PK 
rjKo 



B T PBL 




1. rank(T c 



nandC{E c ) ^ 0; or 



Since B PB is invertible, we have K = (B PB) B PKq. Hence, we can derive the result. 

Lemma 3. £ ^ if and only if there exists E c G ft" x ("-') or E G K" x( - n -' n \ such thatoneofthe 
following conditions holds: 

~ C ' 
E T 

. c . 

2. rank(T = [B E ]) = nandO{E ) j^(Z>. 

where 

C(E C ) = W c f]{(W cl ,W c2 ) : CW cl E c = Q,W c2 E c = 0} 

O(E ) = W„n{(W ol ,W o2 ) : B T W ol E = 0,£ o T W o2 = 0} 
In the affirmative case, any K G £ can be rewritten as 

1. K = W (a C r (CW A C T )- l ;or 

2. K=(B T W ol B)- 1 B T W o2 . 
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Proof: We only prove the statement 2, since the statement 1 is similar. For the necessity, if there 
exist K 6 C, then it shall satisfy Lemma 1. Now we let 

Vi i = P, W o2 = PBK 
Choose E = P Y ,Y = J\f(B ). It is known that [ B E ] is full rank. Then we have 
B T W ol E = B T Y = 0,E T W o2 = YjBK = 

For sufficiency, we assume there exists E such that the statement 2) is satisfied. Notice that 
W ol > and the item W o2 in Y can be rewritten as W ol W ol 1 W 2. 



\-1tT 



Tiat d\-1dTi 



W-^Wol = M W ol T )- L T> W o2 = B(B> W ol B)~ l B l W o2 



(42) 



since T is invertible and B T W iE = 0, E T W 2 = 0. Hence, W -, W o2 can be factorized as 
BK, where K = (B T W ol B)- 1 B T W o2 . Now we can derive (5) from the fact Y < 0. Thus we 
complete the proof. 



hi 







and T„ 1 E 







Similarly, For 



Remark 4. For a given T , since T 1 T = I n ,T 1 B 

a given T c , CT- 1 = [I/O]. 

Theorem 4. C ^ if and only if there exists T c or T , such that one of the following conditions holds: 

1. 

w c ^ 0, Wc = {w cl e n nxn , w c2 e n mxn ■. w cl > o,o c < 0} (43) 

where 



n-l 



A = T c AT,r L , B = T C B,W C 



W cll 
W c22 



and 



w c2 =[w c21 o], w cU eTZ lxl , w c2 2G^ ( " _,)x( " _/) , w c21 en mxl , 

-w cl Aw c1 + bw c2 ] 



*,• 



W cl A T + Wj 2 B T -w cl 



2. 



w ^0, w = {w 0l g n nx ", w o2 e n nxr ■. w 0l > o,o> < o} 



(44) 



where 



and 



A = T-'ATo, C = CT , W ol 



W n 

W o22 



W o2 



W o2 l 





w on eTZ mxm , w o22 G7e("- m ) x ("-'"', w o2 iG^ mxr , 



* D 



-W ol W ol A + W o2 C 

-w 0l 



A T W ol + C T Wj 2 



In the affirmative case, any K e £ caw fee rewritten as 
1. K = W c21 W- 1 \;or 
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2. K 



W£JW 021 . 



Proof: We also only consider the statement 2) here. The sufficiency is obvious according to 
Lemma 3, hence, we only prove the necessity. 
Note that 






-W ol W ol A + W o2 C 

A T W ol + C T Wj 2 -W ol 

-W i W ol A + W o2 C 



A T W ol + C T Wj, -IV, 



\>2 



ol 



To 



where 7^ 



To 
T 



Hence, we can conclude that 



W„i = Tl WoiTo, W 02 = Tl W o2 
Since the system matrices also satisfy 



B T W ol E 



0,E T W o2 







which implies 
Let 



B 1 T- 1 W ol T- L E = 0, E 1 T- 1 W„ 2 = 
W o2 



(45) 



W, 



ol 



W„n W ol2 
Wo T 12 W„ 22 

With the conclusion from Remark 4, (45) implies 



W o2 i 

W o23 



w„ 12 = o, w . 



23 







Hence we have the structural constraints on W„i and W o2 . Using the results of Lemma 3, we 
can easily get the controller L. Thus we complete the proof. 

Remark 5. The first statements of Lemma 3 and Theorem 4 are corollaries of the results in Geromel, 
de Souze & Skelton (1998); Geromel et al. (1996). Based on Theorem 4, we actually obtain a useful 
LMI algorithm for output feedback control design of general LTI systems with fixed E c and/or E . For 
these LTI systems, we can first make a similarity transformation that makes C = [I 0] (or B T = [I 0]). 
Then we force the W c \ and W c2 (or W i and W o2 ) to be constrained structure shown in Theorem 
4. If the corresponding LMIs have solution, we may conclude that the output feedback gain exists; 
otherwise, we cannot make a conclusion, as the choice of E c or E is simply a special case. Thus we 
can choose a scaled E c or E , i.e., eE c or eE to perform a one-dimensional search, which converts 
the LMI condition in Theorem 4 a scaling LMI. For example, <t> c in (43) should be changed as <t> c = 
W lc AW cl + eBW c2 ~ 



W cl A 



T -J- eWlB T 

Cl 



-Wi, 



All the approaches in this section require similarity transformation, which can be done 
by some techniques, such as the singular value decomposition (SVD). However, those 
transformations often bring numerical errors, which sometimes leads to some problems for 
the marginal solutions. Hence in the next section, using Finsler's lemma, we introduce some 
methods without the pretreatment on system matrices. 
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4. Scaling LMIs without similarity transformation 

Finsler's Lemma has been applied in many LMI formulations, e.g., (Boyd et al., 1994; Xu 
et al., 2004). With the aid of Finsler's lemma, we can obtain scaling LMIs without similarity 
transformation. 

Lemma 4. (Boyd et al., 1994) The following expressions are equivalent: 

1. x T Ax > Ofor Mx ^ 0, subject to Bx = 0; 

2. B^AB 1 > 0, where B x ;'s the kernel ofB T , i.e., B ± B T = 0; 

3. A + aB T B > 0,for some scale a G R; 

4. A + XB + B T X T > 0,/or some matrix X. 

In order to apply Finsler's lemma, several manipulation on the Lyapunov inequalities should 
be done first. Note that the condition (5) actually states V(x(t)) = x T (t)Px(t) > and 
AV{x) = V(x(t + 1)) - V(x(t)) < 0. The latter can be rewritten as 



fV£<0, Z=[x T (t)x T (t + l)] T ,V 



-P 
P 



(46) 



Define £ = [x 



T„TiT 



It is easy to verify: 



[K -1]N P £ = 



(47) 
(48) 



where 



M„ 



I 
A B 



,N P 



CO 
I 



(49) 



That is 

(46)s.t.(47)-(48) (50) 

Now based on the statements 1) and 4) of Finsler's Lemma, we can conclude that (50) is 

equivalent to 

- K T- 



M T v VM p 



N> 



1 



X T + X [K -I] N p <0 (51) 



for some X . Now we let 



where £ is a given real scalar, Z 



A" 



[eZ 



? T Z T ] 



y T]T 



(52) 
G Tl mym and Z G Tl nxm . Note that Z is 



constructed from Z with n rows drawing from Z, i.e., Z 

i < m is a vector from Z. Since n > m, there are some same vectors in Z. Now we define 






, zll T , where zT , 1 < 

n j , — 



W = ZK= [w\,wl, 



TiT 



>W m ] 



(53) 



and 



W = ZK=[ 



T T 

W^ , Ifj , 



,,TiT 



(54) 



where wT, 1 < i < m is a vector from W. Then (51) can be transferred into following LMI: 



M T p VM p 



e{C T W T + WC) 
WC - eZ T 



-(Z r + Z) 



< 



(55) 
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Since Z l + Z > B l PB > 0, Z is invertible, K = Z~ l W. 

Theorem 5. The discrete-time system (l)-(2) is stabilized by (3) if there exist P > and Z, W, such 
that (55) is satisfied for some scalar e. Furthermore, the controller is given by K = Z~ l W. 

The conservatism lies in the construction of Z, which has to be a special structure. Z can be 
further relaxed using a transformation Z = eZZ, where Z 6 -j^nxm j g a gj ven ma trbc. In 
Theorem 5, the condition (5) is applied. Based on the condition (6), we have the following 
Lemma. 

Theorem 6. The discrete-time system (l)-(2) is stabilized by (3) if there exist Q > and Z, W, such 

that 

~e{W T B T + BW) * 

W T B T -eZ -{Z T - 



M q QM l 



T _ 



z) 



<0 



(56) 



where 



M a 



I A 
C 



-Q o 

Q 



(57) 



is satisfied for some scalar e. Furthermore, the controller is given by K = Z~ l W. 
Proof. The condition (6) can be rewritten as 

T 



I 



M a QM T q 



Since 



(BK) 1 



(BK) 
-I 



_(BK) T 
0, (58) can be rewritten as 



I 

(BK) T 



<0 



(58) 



(BK) 
-I 



M q QM 



(BK) 
-I 



±T 



<0 



Now applying Finsler's lemma, we have 
M q QM, 



$• 



BK) 
-I 



X + X 



(BK) 
-I 



< 



(59) 



(60) 



for some X = [eZ Z] . Similar to (52), we construct Z from Z with its columns. Hence we have 
(56), which is a sufficient condition for (6). Thus we complete the proof. 

Remark 6. The proof of Theorem 6 is based on the equivalence between 1 and 2 of Finsler's lemma. It 
also provides an alterative proof of Theorem 5 if we note that (5) is equivalent to 



I 
KC 



M T V VM V 



I 
KC 



<0 



(61) 



Remark 7. Except for the case that m = lfor Theorem 5 and I = Ifor Theorem 6, the construction of 
Z is a problem to be considered. So far, we have no systematic method for this problem. However, based 
on our experience, the choose of different vectors and their sequence do affect the result. 

The following simple result is the consequence of the equivalence of 1 and 3 in Finsler's 
Lemma. 
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Theorem 7. The discrete-time system (l)-(2) is stabilized by (3) if there exist P > and K, such that 

< (62) 



-P - eA - eA T + e 2 I A T 
A P- 1 



where e £ 1Z. 



Proof: It is obvious that inequality (42) holds subject to [A — I]£ = 0. Now we apply the 
equivalence between 1 and 3 of Finsler's lemma and obtain 



V-<t\A- 1} 



-1 



-P - aA T A a A 
a A P-crl 



< 



for some a > 0. Note that —A 1 A < -eA 1 - eA + erl, (63) can be implied by 



-P + (r(-eA l - eA + e z I) a A 

a A P-crl 



<0 



(63) 



(64) 



By redefining P as \.P , we can obtain the result. 
Remark 8. Inequality (51) is also equivalent to 



MjVMp - crN? 



for some positive scalar a. Hence, we have 



M T p VM p - N? 



-I 



-1 



[K -I]N P <0 



[K -I] N p <0 



(65) 



(66) 



where V 



-P 
P 



, P = a 1 P. Using the fact that (K — Kq) t (K — Kq) > 0, we may obtain an 



iterative solution from initial condition Kq, where Kq may be gotten from Lemma 5. 

5. Comparison and examples 

We shall note that the comparisons of some existing methods (Bara & Boutayeb, 2005; Crusius 
& Trofino, 1999; Garcia et al., 2001) with the case of e = in Theorem 1 has been given in 
(Bara & Boutayeb, 2006), where it states that there are many numerical examples for which 
Theorem 1 with e = works successfully while the methods in (Bara & Boutayeb, 2005; 
Crusius & Trofino, 1999; Garcia et al., 2001) do not and vice-versa. It also stands for our 
conditions. Hence, in the section, we will only compare these methods introduced above. The 
LMI solvers used here are SeDuMi (vl.3) Sturm et al. (2006) and SDPT3 (v3.4) Toh et al. (2006) 
with YALMIP Lofberg (2004) as the interface. 

In the first example, we will show the advantage of the scaling LMI with £ compared with the 
non-scaling ones. In the second example, we will show that different scaling LMI approaches 
have different performance for different situations. As a by-product, we will also illustrate the 
different solvability of the different solvers. 
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Example 1. Consider the unstable system as follows. 



0.82 0.0576 0.2212 0.8927 0.0678 
0.0574 0.0634 0.6254 0.0926 0.9731 
0.0901 0.7228 0.5133 0.2925 0.9228 
0.6967 0.0337 0.5757 0.8219 0.9587 
0.1471 0.6957 0.2872 0.994 0.5632 



Bo 



0.9505 0.2924 
0.3182 0.4025 
0.2659 0.0341 
0.0611 0.2875 
0.3328 0.2196 



Co 



0.5659 0.255 0.5227 0.0038 0.3608 
0.8701 0.5918 0.1291 0.3258 0.994 



This example is borrowed from (Bara & Boutayeb, 2006), where output feedback controllers 
have been designed. For A22 from A, it has stable eigenvalue. In this paper, we compare the 
design problem with the maximum decay rate, i.e., 



maxjD s.f. 



A T PA 



P < -pP 



Note that in this example, m < n — m. With e = 0, i.e., using the method in (Bara & Boutayeb, 
2006), we obtain the maximum p = 0.16, while Theorem 1 gives p = 0.18 with e = —0.09. 
However, Theorem 5 only obtains a maximum p = 0.03 with a choice of Z = [Ij I2 0] T . 
Note that the solvability heavily depends on the choice of e. For example, when e = 0.09 for 
Theorem 1, the LMI is not feasible. 

Now we consider a case that A22 has an unstable eigenvalue. Consider the above example 
with slight changes on A 



A 



0.9495 0.12048 0.14297 0.19192 0.019139 
0.8656 0.28816 0.67152 0.01136 0.38651 
0.5038 0.46371 0.9712 0.93839 0.42246 
0.13009 0.76443 0.47657 0.54837 0.4089 
0.34529 0.61187 0.15809 0.46639 0.53536 



We can easily verify that A22 from A has one unstable eigenvalue 1.004. Hence, the method 
in (Bara & Boutayeb, 2006) cannot solve it. However, Theorem 1 generates a solution as 

Meanwhile, Theorem 5 also can get a feasible solution for 



K 



£ = -0.1879 and K 
can also obtain K = 



3.61207 0.376493 

"0.93: 



0.4008 
1.5244 -0.7974 
0.3914 -0.3603 
2.3604 -1.1034 



Theorem 4 via a standard SVD without scaling 



using (43) or K 



1.4813 0.5720 
-3.7203 -1.8693 



using (44). 



Example 2. We randomly generate 5000 stabilizable and detectable systems of dimension n 
4(6,6,6,7,7), m = 2(3,1,5,4,3) andl = 2(3,5,1,3,4). 
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Tl 


T3 


SeDuMi 
SDPT3 


5000 
4975 


4982 
5000 



Table 1. Different solvability of different solvers 



Tl" T3 


4.2.2? 


6.3.3 


6.1.5 


6.5.1 


7.4.3 


7.3.4 


Y Y 


4999 


4999 


4994 


4996 


4998 


4998 


Y N 


1 





2 


3 


1 


1 


N Y 





1 


4 


1 


1 


1 


N N 





















Superscript'': Y (N) means that the problem can (not) be solved by the corresponding theorems. For 
example, the value 4 of third row and third column means that in the random 5000 examples, there are 4 
cases that cannot be solved by Theorem 1 while can be solved by Theorem 3. 

Table 2. Comparison of Theorem 1 and Theorem 3 

Hence we can use Theorem 1 and Theorem 3 with e = to solve this problem. Note that 
different solvers may give different solvability. For example, given n = 6, m = 3 and / = 3, 
in a one-time simulation, the result is given in Table 1. Thus in order to partially eliminate 
the effect of the solvers, we choose the combined solvability result from two solvers in this 
section. 

Table 2 shows the comparison of Theorem 1 and Theorem 3. Some phenomenons (the 
solvability of Theorem 1 and Theorem 3 depends on the / and m. When m > /, Theorem 1 
tends to have a higher solvability than Theorem 3. And vise verse.) was observed from these 
results obtained using LMITOOLS provided by Matlab is not shown here. 



6. Extension to Hco synthesis 

The aforementioned results can contribute to other problems, such as robust control. In this 
section, we extend it to Hoo output feedback control problem. Consider the following system: 



x(t + 1) = Ax(t) + B 2 u(t) + B\w 
y[t) = Cx(t) + Div 
z(t) = Ex(t) + Fiv 



(67) 
(68) 
(69) 



We only consider the case that B 2 is with full rank and assume that the system has been 
transferred into the form like (7). Using the controller as (3), the closed-loop system is 



x(t + l) = Ax(t) + Bw 

= {A + B 2 KC)x(t) + (Bj + B 2 KD)w 



(70) 



We attempt to design the controller, such that the L 2 gain sup jpt- < 7. It should be noted 
that all the aforementioned scaling LMI approaches can be applied here. However, we only 
choose one similar to Theorem 1. 
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Theorem 8. The discrete-time system (67)-(69) is stabilized by (3) and satisfies Hoo, if there exist a 
matrix P > defined in (8) and R, such that 



( !R(0i) < 0, m 

I SR(0 2 ) < 0, m < n 

{ 5R(0 3 ) < 0, m > n-m 



n — m 

m 



(71) 



where e €71, 0; is defined in Theorem 1, 



R(0,) = 

P u RC + [P u P 12 ]A RD + [P u P 12 ]B 1 

* A T 0,A-P A T 0,B 1 E T 

* * B T ®jB-yI F T 

* * * — yl 



(72) 



A t pA - P A T PB E T 

* B T PB-yI F T 

* * —yl 



<0 



(73) 



Proof: Following the arguments in Theorem 1, we can see that (71) implies 
R(Gj) 

Using bounded real lemma (Boyd et al., 1994), we can complete the proof. 

7. Conclusion 

In this paper, we have presented some sufficient conditions for static output feedback control 
of discrete-time LTI systems. Some approaches require a similarity transformation to convert 
B or C to a special form such that we can formulate the design problem into a scaling 
LMI problem with a conservative relaxation. Based on whether B or C is full rank, we 
consider several cases with respect to the system state dimension, output dimension and 
input dimension. These methods are better than these introduced in (Bara & Boutayeb, 2006) 
and might achieve statistical advantages over other existing results (Bara & Boutayeb, 2005; 
Crusius & Trofino, 1999; Garcia et al., 2001). The other approaches apply Finsler's lemma 
directly such that the Lyapunov matrix and the controller gain can be separated, and hence 
gain benefits for the design. All the presented approaches can be extended to some other 
problems. Note that we cannot conclude that the approaches presented in this paper is 
definitely superior to all the existing approaches, but introduce some alternative conditions 
which may achieve better performance than others in some circumstances. 
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1. Introduction 

This chapter will consider two discrete time mixed LQR/ H^ control problems. One is the 
discrete time state feedback mixed LQR/ H a control problem, another is the non-fragile 
discrete time state feedback mixed LQR/ H x control problem. Motivation for mixed 
LQR/ H„ control problem is to combine the LQR and suboptimal H^ controller design 
theories, and achieve simultaneously the performance of the two problems. As is well 
known, the performance measure in optimal LQR control theory is the quadratic 
performance index, defined in the time-domain as 



]:=J j {x T {k)Qx(k) + u T (k)Ru{k)) (1) 



T 
I- 

k=0 



while the performance measure in H^ control theory is H x norm, defined in the 
frequency-domain for a stable transfer matrix T zw (z) as 

|T za ,(z)L:= sup <r max [T zw (e ,w )] 

we[0,2n] 

where, Q > , R > , <T max [»] denotes the largest singular value. 

The linear discrete time system corresponding to the discrete time state feedback mixed 

LQR/ H x control problem is 

x(k + 1) = Ax(k) + B lW {k) + B 2 u(k) (2.a) 

z{k)=C t x(k) + D 12 u{k) (2.b) 



with state feedback of the form 



u(k) = Kx(k) (3) 



where, x(k) e R" is the state, u(k) e R m is the control input, w(k) e R' is the disturbance 
input that belongs to L 2 [0,co), z(k) eR p is the controlled output. A, B^ ,B 2 ,C 1 and D 12 
are known matrices of appropriate dimensions. Let x(0) = x . 
The closed loop transfer matrix from the disturbance input w to the controlled output z is 
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T„{Z)' 



M.^|:-C ( u /-.!,) l l\ 



'K i 



where, A K :=A + B 2 K , B K := B t , C K := C a + D 12 K . 

Recall that the discrete time state feedback optimal LQR control problem is to find an 
admissible controller that minimizes the quadratic performance index (1) subject to the 
systems (2) (3) with w = , while the discrete time state feedback H w control problem is to 
find an admissible controller such that ||T ZH ,(z)| < y subject to the systems (2)(3) for a given 
number y > . While we combine the two problems for the systems (2) (3) with w e L 2 [0,oo) , 
the quadratic performance index (1) is a function of the control input u{k) and disturbance 
input w(k) in the case of x(0) being given and y being fixed. Thus, it is not possible to pose 
a mixed LQR/ H m control problem that is to find an admissible controller that achieves the 
minimization of quadratic performance index (1) subject to |T zn ,(z)|| <y for the systems 
(2)(3) with w 6 L 2 [0,oo) because the quadratic performance index (1) is an uncertain function 
depending on the uncertain disturbance input w(k) . In order to eliminate this difficulty, the 
design criteria of state feedback mixed LQR/ H x control problem should be replaced by the 
design criteria 

sup inf {/} subject to I^Jz)^ < y 

weL 2t K 

because for all w e L 2 [0,oo) , the following inequality always exists 

inf {J} < sup inf {J} 

K a>eL 2+ K 

The stochastic problem corresponding to this problem is the combined LQG/ H„ control 
problem that was first presented by Bernstein & Haddad (1989). This problem is to find an 
admissible fixed order dynamic compensator that minimizes the expected cost function of 
the form 

J = limE(x Qx + u Ru) subject to |T_J| <y . 

f->o0 

Here, the disturbance input w of this problem is restricted to be white noise. Since the 
problem of Bernstein & Haddad (1989) involves merely a special case of fixing weighting 
matrices Q and R , it is considered as a mixed H 2 / H^ problem in special case. Doyle et 
al. (1989b) considered a related output feedback mixed H 2 / H„ problem (also see Doyle et 
al., 1994). The two approaches have been shown in Yeh et al. (1992) to be duals of one 
another in some sense. Also, various approaches for solving the mixed H 2 / H^ problem 
are presented (Rotea & Khargonekar , 1991; Khargonekar & Rotea, 1991; Zhou et al., 1994; 
Limebeer et al. 1994; Sznaier ,1994; Rotstein & Sznaier, 1998 ; Sznaier et al. , 2000) . How- 
ever, no approach has involved the mixed LQR/ H m control problem until the discrete time 
state feedback controller for solving this problem was presented by Xu (1996). Since then, 
several approaches to the mixed LQR / H w control problems have been presented in Xu 
(2007, 2008). 

The first goal of this chapter is to, based on the results of Xu (1996,2007), present the simple 
approach to discrete time state feedback mixed LQR / H M control problem by combining 
the Lyapunov method for proving the discrete time optimal LQR control pro-blem with an 
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extension of the discrete time bounded real lemma, the argument of compl-etion of squares 

of Furuta & Phoojaruenchanachi (1990) and standard inverse matrix man-ipulation of Souza 

&Xie(1992). 

On the other hand, unlike the discrete time state feedback mixed LQR / H x control 

problem, state feedback corresponding to the non-fragile discrete time state feedback mixed 

LQR/ H„ control problem is a function of controller uncertainty AF(fc), and is given by 



u(k) = F a3 x(k),F a0 



-AF(fc) 



(4) 



where, AF(fc) is the controller uncertainty. 

The closed-loop transfer matrix from disturbance input w to the controlled output z and 

quadratic performance index for the closed-loop system (2) (4) is respectively 



T zw(z)-- 



-c-(zi^A,y 



and 



/:=S 



Q /2 x{k) 



R/ 2 u{k) 



-y 2 \ w \ 1 



where, Ap 



■A + B^, 



• C t 



- Cj + D 12 F m , y > is a given number. 



Note that the feedback matrix F w of the considered closed-loop system is a function of the 
controller uncertainty AF(fc) , this results in that the quadratic performance index (1) is not 
only a function of the controller F w and disturbance input w(k) but also a function of the 
controller uncertainty AF(fc) in the case of x(0) being given and y being fixed. We can 
easily know that the existence of disturbance input w(k) and controller uncertainty AF(/c) 
makes it impossible to find sup a , sL inf K {/} , while the existence of controller uncertainty 
AF(fc) also makes it difficult to find sup^,^ {/} . In order to eliminate these difficulties, the 
design criteria of non-fragile discrete time state feedback mixed LQR/ H x control problem 
should be replaced by the design criteria 

sup H , eL2+ {/} subject to |T za ,(z)|| oo < r . 

Motivation for non-fragile problem came from Keel & Bhattacharyya (1997). Keel & 
Bhattacharyya (1997) showed by examples that optimum and robust controllers, designed 
by using the H 2 , H^ , I , and jj. formulations, can produce extremely fragile controllers, in 
the sense that vanishingly small perturbations of the coefficients of the designed controller 
destabilize the closed-loop system; while the controller gain variations could not be avoided 
in most applications. This is because many factors, such as the limitations in available com- 
puter memory and word-length capabilities of digital processor and the A/D and D/A 
converters,result in the variation of the controller parameters in controller implementation. 
Also, the controller gain variations might come about because of external effects such as 
temperature changes. Thus, any controller must be insensitive to the above-mentioned con- 
toller gain variation. The question arised from this is how to design a controller that is inse- 
nsitive, or non-fragile to error/ uncertainty in controller parameters for a given plant. This 
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problem is said to be a non-fragile control problem. Recently, the non-fragile controller 
approach has been used to a very large class of control problems (Famularo et al. 2000, 
Haddad et al. 2000, Yang et al 2000, Yang et al. 2001 and Xu 2007). 

The second aim of this chapter is to, based on the results of Xu (2007), present a non-fragile 
controller approach to the discrete-time state feedback mixed LQR/ H w control problem 
with controller uncertainty. 

This chapter is organized as follows. In Section 2, we review several preliminary results, and 
present two extensions of the well known discrete time bounded real lamma. In Section 3, 
we define the discrete time state feedback mixed LQR/ H^ control problem. Based on this 
definition, we present the both Riccati equation approach and state space approach to the 
discrete time state feedback mixed LQR/ H m control problem. In Section 4, we intro-duce 
the definition of non-fragile discrete time state feedback mixed LQR/ H m control problem, 
give the design method of a non-fragile discrete time state feedback mixed LQR / H x 
controller, and derive the necessary and sufficient conditions for the existence of this 
controller. In Section 5, we give two examples to illustrate the design procedures and their 
effectiveness, respectively. Section 6 gives some conclusions. 

Throughout this chapter, A denotes the transpose of A , A~ denotes the inverse of A , 
A~ is the shorthand for (A~ ) , G~(z) denotes the conjugate system of G(z) and is the 
shorthand for G (zT ),L 2 (-co,+co) denotes the time domain Lebesgue space, L 2 [0,+°o) 
denotes the subspace of L 2 (-co,+co) , L 2 (-°°,0] denotes the subspace of L 2 (-co,+co), L 2+ is 
the shorthand for L 2 [0,+co) and L 2 _ is the shorthand for L 2 (-<x>,0]. 

2. Preliminaries 

This section reviews several preliminary results. First, we consider the discerete time Riccati 
equation and discrete time Riccati inequality, respectively 

X = A T X(I + RXy 1 A + Q (5) 

and 

A T X(I + RXy 1 A+Q-X<0 (6) 

with Q = Q T > and R = R T > . 

We are particularly interested in solution s X of (5) and (6) such that (I + RX)~ A is stable. 
A symmetric matrix X is said to the stabilizing solution of discrete time Riccati equation (5) 
if it satisfies (5) and is such that (I + RX)~ A is stable. Moreover, for a sufficiently small 
constant S > , the discrete time Riccati inequality (6) can be rewritten as 

X = A T X(I + RXy 1 A + Q + SI (7) 

Based on the above relation, we can say that if a symmetric matrix X is a stabilizing 
solution to the discrete time Riccati equation (7), then it also is a stabilizing solution to the 
discrete time Riccati inequality (6). According to the concept of stabilizing solution of 
discrete time Riccati equation, we can define the stabilizing solution X to the discrete time 
Riccati inequality (6) as follow: if there exists a symmetric solution X to the discrete time 
Riccati inequality (6) such that (I + RX)~ A is stable, then it is said to be a stabilizing 
solution to the discrete time Riccati inequality (6) . 
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If A is invertible, the stabilizing solution to the discerete time Riccati equation (5) can be 
obtained through the following simplectic matrix 



A + RA- T Q 
-A- T Q 



-RA~ 



(8) 



Assume that S has no eigenvalues on the unit circle, then it must have n eigenvalues in 
Uf <1 and win U,|>1 (i = 1,2,---, n,n + l,---,2n ). If n eigenvectors corresponding to n 
eigenvalues in U,-| < 1 of the simplectic matrix (8) is computed as 

u i 
then a stabilizing solution to the discerete time Riccati equation (5) is given by 

X = [Pi ■■■ P»][mi •■■ «„]" 

Secondly, we will introduce the well known discrete time bounded real lemma (see Zhou et 
al. , 1996; Iglesias & Glover, 1991; Souza & Xie, 1992) . 
Lemma 2.1 (Discrete Time Bounded Real Lemma) 

"A - . , 

; RH X , then the following two statements are 



Suppose that y>0, M(z) - 



C \D 



equivalent: 

i- \M4l<r ■ 

ii. There exists a stabilizing solution X > ( X > if (C,A) is observable ) to the discrete 
time Riccati equation 

A T XA -X + y~ 2 (A T XB + C T D)U r 7 1 (B T XA + D T C) + C T C = 

such that U t = I - y- 2 (D T D + B T XB) > . 

In order to solve the two discrete time state feedback mixed LQR/ H^ control problems 

considered by this chapter, we introduce the following reference system 



x(k + l) = Ax{k) + B 1 w{k) + B 1 u(k) z{k) 



n^ 



*(*)+ 



D, 



u(k) 



(9) 



where, CI - 



Q o 

R 



and z(k) ■■ 



z(k) 



The following lemma is an extension of the discrete time bounded real lemma. 

Lemma 2.2 Given the system (2) under the influence of the state feedback (3), and suppose 

that y>0, T zw (z)eRH w ; then there exists an admissible controller K such that 

||T ZH ,(z)| <y if there exists a stabilizing solution X„ > to the discrete time Riccati 

equation 



A T K X X A K -X x + y- 2 A T K X„B K U?BlX x A K + C T K C K +Q + K T RK = 



(10) 
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W K >o. 



such that U-y = I - y~ 2 Bp 

Proof: Consider the reference system (9) under the influence of the state feedback (3), and 



define T as 



W:= 



A 


K 


Bit] 


& 


3 






then the closed-loop transfer matrix from disturbance input w to the controlled output z is 



ZW ^ Z> I Note that y 2 I - F~ ZW F~ ZW > is equivalent to 



T {z) 



rl - T;j zw > T ~T > for all w s L 2 [0,=c) , 

and T zw( z ) eRH «,is equivalent to T ia (z)eRH txl , so l^^L < y implies \T zw (z)\ fj <y 
Hence, it follows from Lemma 2.1. Q.E.D. 

To prove the result of non-fragile discrete time state feedback mixed LQR/ H ra control 
problem, we define the inequality 



Aj X w A p -X. + y- 2 A T p X m B p U?B T p X m A, + C T p C p + Q+PjRF m < 



(11) 



where, U 1 =I- y' 2 BJ X«,B p > . 

In terms of the inequality (11), we have the following lemma: 

Lemma 2.3 Consider the system (2) under the influence of state feedback (4) with controler 
uncertainty, and suppose that y > is a given number, then there exists an admissible non- 
fragile controller F„ such that ||T zn ,| <y if for any admissible uncertainty AF(fc), there 
exists a stabilizing solution X„ > to the inequality (11) such that LZ X = 
I-y- 2 B\X„B t >0. 

Proof: Suppose that for any admissible uncertainty AF(fc), there exists a stabilizing solution 
X„ > to the inequality (11) such that U x = I - y~ 2 BJ X K Bp > . This implies that the 
solution X^ > is such that A p + y~ 2 Bp U^B p X M A F is" stable. 
Let A F = A + B 2 F a0 and C F = Cj + F>i 2 F m ; then we can rewrite (11) as 



AlXA 



•X„ + r- 2 A£xjB*U7 1 BjX„A l! 



-clc T 



■Q 



F„ oo F m oo / Ftf) <» F 1 P oc > F c v 'F :o ^'F o:i 

+ F^RF m - (A r LZ 3 B 2 + FlU 2 )U 2 \B T 2 U 3 A + U 2 F m ) + AN F < 



where, U 2 = B l 2 U 3 B 2 +I + R, U 3 = y~ 2 X m B p U?B' p X w + X m , 
AN" F = (A T U 3 B 2 + FjL7 2 + AF r (fc)U r 2 )U r 2 1 (B 2 r L/ 3 A + U 2 F„ + U 2 AF(k)) . 

Since AF(k) is an admissible norm-bounded time- varying uncertainty, there exists a time- 
varying uncertain number S(k) > satisfying 



Ap X^Ar 



- X„ + y- 2 A\ X X B L U?Bl X X A^ + c£c F . + Q + f£rF„„ 

- (A T U 3 B 2 + FlU 2 )U?{BlUi,A + U 2 F X ) + AN F + S(k)I = 



(12) 
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Note that Ap + y~ Bf, U{ B- X x Ap is stable for any admissible uncertainty AF(k) . This 
implies that A F + y~ 2 Bp U^BJ X X A F is stable. 

Hence, (L7j B~ X^Ap ,A F ) is detectable. Then it follows from standard results on 
Lyapunov equations (see Lemma 2.7 a), Iglesias & Glover 1991) and the equation (12) that 
A F is stable. Thus, A= = A F + B 2 AF(k) is stable for any admissible uncertainty AF(k) . 
Define V(x(k)) := x (k)X ao x(k) , where, x is the solution to the plant equations for a given 
input w , then it can be easily established that 

= X {~ A v ( x ( k )) + x T (k + l)X x x(k + l)-x T (fc)X„x(fc)} 



= X{-AV(z(fc))-||z|| +r 2 |H 

fc=0 



- 2,L..„2 y 2 



U{ 2 (w- r - z U?B F X x A t x) 
+X 1 (Ai_ X„A t - X K + r ^ X„B t U^BT X„A^ + CJ[C # _ )x] 
Add the above zero equality to / to get 



J = Z f- Ay M fc )) - INf + t' 2 IH 

Jc=0 



|2 2 

-r 



ufi(w-r-*u?Blx„Apx) 



+x T (AJX^ -X ai + y- 2 Aj_X 00 B # _Ur 1 Bj_X 00 A f<i +Cj_C f _ + Q + FJRF.)*} 
Substituting (11) for the above formula,we get that for any u(k) and if(A:) and x(0) = , 

2 



r II II 2 2 || ||2 2 



U^(w-y- 2 UT 1 BjX B) A # *) 



Note that |z | 2 = ^x T (A:)Qx(A:), and define that r:=w-y 2 LT 1 1 B| X^Ap x , we get 

ic=0 



-^ to L <-y 



lf{ 2 r 



Suppose that F is the operator with realization 

x(fc + l) = (A + B 2 F c )x(fc) + B f w(k) 

r(k) = -r' 2 U^BJ X X A P x(k) + w(k) 

which maps w to r . 

Since T' 1 exists ( and is given by x(k + 1) = (A + B 2 F m + y~ 2 B p U^BJ X m A f )x(k) + B p r(k) , 

w(k) = y^U^BJ X„A p x(k) + r(k) ), we can write 



■y to <-y 



U( 2 r 



2 Hi- II 2 ^ II II 2 

-y Fh; <*- to 
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for some positive k .This implies that there exists an admissible non-fragile controller such 
that llTjjJ < y ■ Note that y I - T^T^ W > is equivalent to 

r 2 I - T~ a T za > T~T > for all w e L 2 [0,co) 

so \\T~ II <y implies ||T_„| < y , and we conclude that there exists an admissible non-fraeile 
controller such that TL, < y . Q. E. D. 

II 2tf||oo / ^~ 

3. State Feedback 

In this section, we will consider the discrete time state feedback mixed LQR/ H x control 
problem. This problem is defined as follows: Given the linear discrete-time systems (2) (3) 
with we L 2 [0,co) and x(0) = x and the quadratic performance index (1), for a given 
number y >0, determine an admissible controller K that achieves 

sup inf {/) subject to fr^z)^ < y . 

weL 2+ K 

If this controller K exists, it is said to be a discrete time state feedback mixed LQR/ H x 

controller. 

Here, we will discuss the simplified versions of the problem defined in the above. In order 

to do this, the following assumptions are imposed on the system 

Assumption 1 (C 1 ,A) is detectable. 

Assumption 2 (A,B 2 ) is stabilizable. 

Assumption3 D 12 [Cj D 12 ] = [0 I] . 

The solution to the problem defined in the above involves the discrete time Riccati equation 

A T X X A - X. - A T XJ(B T XJ + R)- 1 B r X o0 A + Cfo + Q = (13) 



If A is invertible, the stabilizing solution to the 



-I 
R + I 
discrete time Riccati equation (13) can be obtained through the following simplectic matrix 



where, B^y-^ B 2 ], R 



'A + BR~ 1 B T A~ T (d[C 1 +Q) -BR~ 1 B T A~ r 

-A- T (CjC 1+ Q) A- T 

In the following theorem, we provide the solution to discrete time state feedback mixed 
LQR/ H K control problem. 

Theorem 3.1 There exists a state feedback mixed LQR/ H M controller if the discrete time 
Riccati equation (13) has a stabilizing solution X^ > and U 1 =I - y' 2 B^X a0 B 1 > . 
Moreover, this state feedback mixed LQR/ H x controller is given by 

K = -U r 2 1 B 2 r L7 3 A 
where, U 2 = R + 1 + B T 2 U 3 B 2 , and U 3 =X al + y^X^BjU^BjX^ . 
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In this case, the state feedback mixed LQR/ H„ controller will achieve 

sup inf{ /} = x T (X„ + y- 2 X w - X z )x subject to ||T„L < y . 

weL 2+ K 

where, A K = A K + y- 2 B K U^B T K X x A K , X w = f j {(A k K ) T A T K X a: B ¥ U^BlX x A K A k K }, and 

Jc=0 

00 

X z =Y{{Al) T C T K C K Al}. 

Before proving Theorem 3.1, we will give the following lemma. 

Lemma 3.1 Suppose that the discrete time Riccati equation (13) has a stabilizing solution 

X M > and U x =l- y^BjX^ > , and let A K =A + B 2 K and K = -U?B T 2 U 3 A ; then A K is 

stable. 

Proof: Suppose that the discrete time Riccati equation (13) has a stabilizing solution X„ > 

and U 1 = I - y^BjX^ > . Observe that 

X^y-% B 2 ]- 



B'X^B + R- 






-I 

R + I 



-U, 



r" 1 Bi r x 00 B 2 



y^BlX^ B T 2 X x B 2 +R + l 



Also, note that U X =I- y^B^X^ >0,U 3 =X^+ y~ 2 X x B^Jy B{X , and U 2 = R + 1 
+B 2 U 3 B 2 ; then it can be easily shown by using the similar standard matrix manipulations as 
in the proof of Theorem 3.1 in Souza & Xie (1992) that 

-Uf 1 + U^B-yU^BjUi 1 U^BJJz 
U^BjU? L7 2 : 



{FxJ + Ry 1 



where, B t = ^" 1 B[X 00 B 2 . 
Thus, we have 

A T Xj(B T Xj + R^^X^ = -y-^X^U^BjX^A + A T U 3 B 2 U?B T 2 U 3 A 
Rearraging the discrete time Riccati equation (13), we get 



X„ = A T X X A + y- 2 A T X 00 B 1 U^BlX ao A - A T U r 3 B 2 ii 2 1 B 2 r U r 3 A + Cfci + Q 

= A T X„A + r-WX&UfBlXJi + Cfo +Q- A T U 3 B 2 L/ 2 1 B 2 r (X 0O + y^X^U^BjXJA 

- A r (X„ + r - 2 X o0 B 1 Ur 1 B 1 r X o0 )B 2 L7 2 1 B 2 T Li 3 A 

+ A T U 3 B 2 U 2 1 [R + 1 + B 2 r (X„ + r - 2 X o0 B 1 L7 1 - 1 B 1 r X OD )B 2 ]U 2 1 B 2 T !J 3 A 
= ( A T X a0 A - A T U 3 B 2 U r 2 1 B 2 r X os A - A r X os B 2 U r 2 1 B 2 r LZ 3 A + A r L7 3 B 2 LI 2 1 B 2 r X 0O B 2 LI 2 1 B 2 r U 3 A) 

+ (ClC 1 + A T U 3 B 2 U 2 1 U 2 1 B T 2 U 3 A) + A T U 3 B 2 U 2 1 RU 2 1 B T 2 U 3 A + Q 

+ (y- 2 A T X as B 1 Ul 1 BlX x A - r- 2 A T U 3 B 2 Li 2 1 B 2 r X o= B 1 L7 1 - 1 B 1 r X 00 A 

- y- 2 A T X„p l U^B T 1 X„ J B 2 U 2 1 B T 2 U 3 A + /" 2 A r L7 3 B 2 LI 2 1 B 2 r X 0O B 1 L/ ] ; 1 B 1 T X OD B 2 L/ 2 1 B 2 r U r 3 A) 
= ( A - B 2 L7 2 1 B 2 r U 3 A) r X 00 ( A - B 2 L7 2 1 B 2 r U 3 A) + (C : - D^U^BjUj A) r (C a - D X2 U?B T 2 U 3 A) 

+ K r RK + Q + y- 2 (A - B 2 L7 2 1 B 2 r U r 3 A) T X oo B 1 !J 1 ; 1 B 1 r X 0O (A - B^Bjl^A) 
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that is, 



A T K X m A K - X m + y- 2 AlXJ$ K U?BlX x A K + C T K C K +Q + K T RK = 



(14) 



Since the discrete time Riccati equation (13) has a stabilizing solution X K > , the discrete 
time Riccati equation (14) also has a stabilizing solution X^ > . This implies that 
A K =A K +y~ 2 B K U{ 1 B T K X x A K is stable. Hence (Ui 1 B^X ay A K ,A K ) is detectable. Based on this, 
it follows from standard results on Lyapunov equations (see Lemma 2.7 a), Iglesias & 
Glover 1991) that A K is stable.Q. E. D. 

Proof of Theorem 3.1: Suppose that the discrete time Riccati equation (13) has a stabilizing 
solution X^ >0 and U 1 = 1 -y' 2 B^X x B 1 >0 . Then, it follows from Lemma 3.1 that A K is 
stable. This implies that T zw (z) e RH m . By using the same standard matrix manipulations as 
in the proof of Lemma 3.1, we can rewrite the discrete time Riccati equation (13) as follows: 

A T X X A -X m + y- 2 A T X ao B 1 U^ 1 BjX ao A - A T U 3 B 2 U 2 1 BlU 3 A + Cfo + Q = 
or equivalently, 

A T K X m A K - X„ + y- 2 A T K X x B K U?B T K X„A K + C T K C K +Q + K T RK = 

Thus, it follows from Lemma 2.2 that |T 2M ,(z)| < y ■ 

Define V(x(k)) = x (k)X x x(k) , where X m is the solution to the discrete time Riccati equation 

(13), then taking the difference AV(x(k)) and completing the squares we get 

AV(x(k)) = x T (k + l)X m x{k + 1) - x T (k)X m x(k) 

= x T (k)(A T K X rr A K - XJx(k) + x T (k)A T K X rx B K w(k) 
+ w J \k)B T K X ra A K x{k) + w T (k)B T K X^B K w(k) 



II II 2 2 || ||2 2 

■ -z +y \\w\\ -y 



U{ 2 (w-y- 2 U?B T K X m A K x) 



+ x T (A T K X x A K -X m+r - 2 A T K X x B K U?B T K X x A K +C T K C K )x 



Based on the above, the cost function / can be rewritten as: 



] = Y i x T (mx(k) = Z{-AV(x(k))- 

k=0 k=0 



II II 2 2 || ||2 2 

z +y \\w\\ -y 



U( 2 {w-y- 2 U?B l K X x A K x) 



(15) 



+ x' {A 1 ti X ai A K -X^+y- l A 1 K X^ K U- 1 l B l K X VJ A K +C l ¥ C K +Q+K 1 RK)x} 



On the other hand, it follows from the similar argumrnts as in the proof of Theorem 3.1 in 
Furuta & Phoojaruenchanachai (1990) that 

A T K X X A K - X K + y- 2 AlX x B K U?BlX x A K + C T K C K +Q+ K T RK 
= A T X a A -X m + r-Wx&UfBZXJL - A T L? 3 B 2 U 2 1 B 2 r U 3 A + cjc a + Q 
+ (K + U^BlU z A) T U 2 {K + U?BlU 3 A) 



At the same time note that 
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We have 



-y- 2 A T X rt fi 1 U^BlX ra A + A T U i B 2 U 2 1 B T 2 U 3 A 



-A T XJ 



-L/j 1 + U^B^BlU^ U^B-jUj 1 



u^bIu: 1 



u; 



B T XA 



= A T Xj(8 r Xj + R)~ 1 8 7 XA 



A T K X a0 A K - X„ + y- 2 A T K X x B K U?BlX ra A K + C T K C K +Q + K T RK 
= A T X X A - X, - A T XJ(B T XJ + A)- 1 B r X o0 A + Cfo + Q 
+ (K + U 2 1 B 2 r U 3 A) T U 2 (K + U^BlU-s A) 

Also, noting that the discrete time Riccati equation (13) and substituting the above equality 
for (15), we get 



/ = JT x T (k)Ox(k) = ]T {-AV(x(k)) - \\zf + y 2 \\wf - y 2 



UP(K + U 2 B 2 U 3 A)x 



U( 2 {w-y- 2 U?BlX x A K x) 



(16) 



Based on the above, it is clear that if K = -U 2 B 2 U 3 A , then we get 



inf {/} = Xq X x x - \\z\\ 2 + y 2 \\w\\ 2 - y 2 



U( 2 {w~y- 2 U?BlX ra A K x) 



(17) 



By letting w(k) = y 2 LZ 1 1 BkX 00 A k x(J:) for all k > , we get that x{k) = A k K x with A K which 
belongs to L 2 [0,+oo) since A K = A-B(B T X X B + ky 1 B T X a0 A is stable. Also, we have 



H*)|£ =r- i x T X a x ,\\z(k)g=x T X z x 
Then it follows from (17) that 

sup inf {/} = x£(X m + y- 2 X w - X z )x 

iceL 2+ 

Thus we conclude that there exists an admissible state feedback controller such that 



sup inf {/} = x T (X m + y- 2 X w - X z )x subject to ||T OT L < y 



Q.E.D. 



4. Non-fragile controller 

In this section, we will consider the non-fragile discrete-time state feedback mixed 
LQR/ H K control problem with controller uncertainty. This problem is defined as follows: 
Consider the system (2) (4) satisfying Assumption 1-3 with w e L 2 [0,co) and x(0) = x , for a 
given number y > and any admissible controller uncertainty, determine an admissible 
non-fragile controller F m such that 



170 



New Trends in Technologies 



sup {J} subject to \r zw {z)\ a <r . 

wgL 1+ 

where, the controller uncertainty AF(fc) considered here is assumed to be of the following 
structure: 

AF(k) = H K F(k)E K 

where, H K and E K are known matrices of appropriate dimensions. F(k) is an uncertain 
matrix satisfying 

F T (k)F{k)<I 

with the elements of F(k) being Lebesgue measurable. 

If this controller exists, it is said to be a non-fragile discrete time state feedback mixed 

LQR/ H„ controller. 

In order to solve the problem defined in the above, we first connect the its design criteria 

with the inequality (11). 

Lemma 4.1 Suppose that y > , then there exists an admissible non-fragile controller F„ 

that achieves 

sup {/} = XqXjXq subject to {{T^ < y 

w=L 2+ 

if for any admissible uncertainty AF(fc), there exists a stabilizing solution X„ > to the 
inequality (11) such that U x = I - y~ 2 Bp X x Bp > . 

Proof: Suppose that for any admissible uncertainty AF(fc), there exists a stabilizing solution 
X x > to the inequality (11) such that U 1 =I- y~ 2 BJ X^Bp > . This implies that the 
solution X^ > is such that Ap + y~ Bp lij Bp X^Ap is stable. Then it follows from 
Lemma 2.3 that ||F za ,| < y . Using the same argument as in the proof of Lemma 2.3, we get 
that Ap is stable and / can be rewritten as follows: 



/ = £ {-AV(x(k)) - \\zf + y 2 \\w( - y 7 



U{ 2 {w-y- 2 U?B T pX m Apx) 



k=o (18) 

+x T (A T pX a3 Ap -X^ + y^AjX^BpUMX^Ap +C T pC p +Q + F^RFJx} 



X ™ F x 

Substituting (11) for (18) to get 



/ < XoX*o - INL + r 2 |w|L - y 2 



UPiw-r^UfBJXJipx) 



(19a) 



]<xlX w x -\\zf-y 2 



ufc{w-y- 2 U?B T pX x Apx) 



(19b) 



By letting w = y 2 LT 1 1 Bj X^Ap x for all k>0, we get that x(k) = A k p x with 



F x ™ F„ 



Ap = Ap +y 2 Bp U 1 1 BJ X x Ap which belongs to L 2 [0,+co) since Ap is stable. It follows 
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from (19b) that sup{/} , = xj. X x x ■ Thus, we conclude that there exists an admissible 
non-fragile controller such that supf/lj,,^ = XqX oo x subject to \\T zw \\ rj:> < y . Q. E. D. 
Remark 4.1 In the proof of Lemma 4.1, we let w = y~ M\ Bp X^Ap x for all k > to get that 
x(k) = ApX with Ap = Ap + y~ 2 Bp U^Bp X^Ap which belongs to L 2 [0,+oo) since Ap is 
stable. Also, we have 



112 

Then it follows from (19a) that 



i|2 _4 Tv II l|2 Tv 

Mu=r x x w x ,\\z\\ =X X Z X . 



]<x T (X x+ y- 2 X w -X z )x (20) 



where, X w = £{(A* f A j XJjtfBfXJLp A\ } , and X z = £{(A) fcj Cp A k p } . 

k=0 " k=0 " " * 

Note that Ap depends on the controller uncertainty AF(fc), thus it is difficult to find an 
upper bouncf of either of X w and X z . This implies that the existence of controller 
uncertainty AF(k) makes it difficult to find sup H , sL {/} by using (20). Thus, it is clear that 
the existence of the controller uncertainty makes the performance of the designed system 
become bad. 

In order to give necessary and sufficient conditions for the existence of an admissible non- 
fragile controller for solving the non-fragile discrete-time state feedback mixed LQR/ H x 
control problem, we define the following parameter-dependent discrete time Riccati 
equation: 

A T X m A-X x -A T xJ{B T xJ + R)- 1 B T X^A + p 2 ElE K +CjC 1+ Q s =0 (21) 



where, B = [r _1 B a B 2 ],R = 



-I 
I + R 



i Qs =Q + SI with S > being a sufficiently small 



constant, p is a given number satisfying pi- H K U 2 H K > , U x = I - y B r X a3 B 1 > , 
U 2 = BlU 3 B 2 +I + R and U 3 = X K + /^X^U^BjX, . If A is invertible, the parameter- 
dependent discrete time Riccati equation (21) can be solved by using the following 
symplectic matrix 

A + BR- 1 B T A- T (p 2 ElE K +C[C a +Q S ) -BR" 1 B T A" T 

-A- J (p 2 E T K E K+ ClC l+ Q s ) A- T 

The following theorem gives the solution to non-fragile discrete time state feedback mixed 
LQR/ H x control problem. 

Theorem 4.1 There exists a non-fragile discrete time state feedback mixed LQR/ H x 
controller iff for a given number p and a sufficiently small constant S > , there exists a 
stabilizing solution X x > to the parameter-dependent discrete time Riccati equation (21) 
such that U a = I - y^X^ > and p 2 I - H T K U 2 H K > . 
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Moreover, this non-fragile discrete time state feedback mixed LQR/ H„ controller is 

F a0 =-U 2 1 B T 2 U 3 A 

and achieves Sup{/} rosL2+ = x T X^ subject to {{T^ < y . 

Proof: Sufficiency: Suppose that for a given number p and a sufficiently small constant 
8 > , there exists a stabilizing solution X^ > to the parameter-dependent Riccati 
equation (21) such that U a = I - y'^X^ > and p 2 l - H T K U 2 H K > . This implies that the 
solution X„>0 is such that A-B(B T X m B + ky 1 B T X a0 A is stable. Define respectively the 
state matrix and controlled output matrix of closed-loop system 

At. = A + B 2 (-U 2 1 BlU 3 A + H K F(k)E K ) 



C P = C 1+ D 12 (-U 2 1 B T 2 U 3 A + H K F(k)E K ) 



and let A F = A - B^B^A and F^ = -U?B\U 3 A + H K F(k)E K , then it follows from the 
square completion that 

AJX x A t -X a + y- 2 AlX m B p U?B T p X m A t +CJC t +Q + FjRF x 

= A T X X A - X K + y- 2 A T X x B 1 UtB T 1 X x A + Cfo + Q + F^B T 2 U 3 A + A T U 3 B 2 F X + FjU 2 F x (22) 
= A T X al A -X„ + y- 2 A T X ao B 1 Ul 1 BlX ao A + Cfo + Q - A T U 3 B 2 U 2 1 B 2 r iJ 3 A + AN 

where, AN = E T K F T {k)H J K U 2 H K F{k)E K . 
Noting that p 2 I - H T K U 2 H K > , we have 

m = -E T K F T (k)(p 2 I-H T K U 2 H K )F(k)E K +p 2 E T K F T (k)F(k)E K < p 2 E T K E K (23) 

Considering (22) and (23) to get 

AJX^ -X m +y- 2 AlX„B~U?BlX„A t +Cj_C f _ + Q + F^RF X 

< A T X X A - X. + y-^X^U^X^A + Cfo + Q + p 2 E T K E K - A T U 3 B 2 U 2 1 B 2 r !J 3 A 



(24) 



Also, it can be easily shown by using the similar standard matrix manipulations as in the 
proof of Theorem 3.1 in Souza & Xie (1992) that 

A T Xj(B T Xj + RyWx^A = - r -*A T X ao B 1 U?B[X m A + A T U 3 B 2 L7 2 1 B 2 r L/ 3 A 
This implies that (21) can be rewritten as 

A T X X A -X m + r-*A T XJBJI?BZXJl. + Cfo + Q s - A T L7 3 B 2 U 2 1 B 2 r U 3 A + p 2 E T K E K = (25) 

Thus, it follows from (24) and (25) that there exists a non-negative-definite solution to the 
inequality 

Aj X x A p - X„ + y- 2 A T p X x B p UM X x A p + C T p C p +Q + FjRF m < 



Discrete Time Mixed LQR/hL Control Problems 1 73 

Note that A-B{B T Xji + R)~ 1 B r X os A = A F + y^B^B^X^A^ is stable and AF(k) is an 

admissible uncertainty, we get that As + y~ Bp L7j Bp X rr Ap is stable. By Lemma 4.1, there 

exists a non- fragile discrete time state feedback mixed LQR/ H x controller. 

Necessity: Suppose that there exists a non-fragile discrete time state feedback mixed 

LQR/ H K controller. By Lemma 4.1, there exists a stabilizing solution X^ > to the 

inequality (11) such that Uj = I — y~ Bp X^Bp > , i.e., there exists a symmetric non- 
negative-definite solution X^ to the inequality (11) such that Ap + y~ Bp U^ Bp X x Ap is 
stable and U 1 = I - y~ Bp X x Bp > for any admissible uncertainty AF(k) . 
Rewriting (11) to get 



A£ X„ A Fx - X„ + y- 2 A£ XJSjU^B^XJi.^ + C£ C F ^+Q + F?RF W + AN < 
AN" = (A T U 3 B 2 + F£u 2 )AF(k) + AF T (fc)(BjU 3 A + U 2 FJ + AF T (k)U 2 AF(k) 

Note that p 2 I - H T K U 2 H K > and 

AN = p 2 E T K F T (k)F(k)E K +(A T U 3 B 2 +F^U 2 )H K (p 2 I-H T K U 2 H K y 1 H T K 

x(B T 2 U 3 A + U 2 FJ-((A T U 3 B 2 + Fj j U 2 )H K (p 2 I-H T K U 2 H K )- 1 -E T K F T (k)) 
x(p 2 I-HlU 2 H K )(( P h-HlU 2 H K y l H T K (B T 2 U 3 A + U 2 FJ-F(k)E K ) 
<p 2 E T K E K +(A T U 3 B 2 +F^U 2 )H K (p 2 I-H T K U 2 H K )- 1 H T K (B T 2 U 3 A + U 2 FJ 

It follows from (26) and (27) that 

AlX ra A K -X^+y^AlX^U^BlX^ +C£C F . + Q + F^RF^ + p 2 E T K E K 
+(A T U 3 B 2 + FlU 2 )H K {ph-HlU 2 H K )- x Hl{B T 2 U 3 A + U 2 F m ) < 



(26) 



(27) 



(28) 



Using the argument of completion of squares as in the proof of Theorem 3.1 in Furuta & 
Phoojaruenchanachai (1990), we get from (28) that F m = -U^B^U^ , where X„ is a 
symmetric non- negative-definite solution to the inequality 

A T X X A - X, + y-^X^U-^X^A + cfo + Q - A J U 3 B 2 U?B T 2 U 3 A + p 2 E T K E K < 

or equivalently, X^ is a symmetric non-negative-definite solution to the parameter- 
dependent discrete time Riccati equation 

A T X a A -X m + y-^X^U^X^A + c[C a + Q s - A J U 3 B 2 UtB T 2 U 3 A + p 2 E T K E K = (29) 
Also, we can rewrite that Riccati equation (29) can be rewritten as 

A T X m A-X x -A T xJ{B T xJ + Ry l B J X„A + p 2 ElE K +C J 1 C 1 +Q s =0 (30) 
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by using the similar standard matrix manipulations as in the proof of Theorem 3.1 in Souza 
& Xie (1992). Note that A - B(B J xji + ky 1 B J X X A = A F ^ + r^B-yU^BjX^A^ and AF(k) is 

an admissible uncertainty, the assumption that At, + y~ Bp !ij Bp X^Ap is stable implies 

that A-B(B X X B + R)~ B X X A is stable Thus, we conclude that for a given number p and 
a sufficiently small number S > , the parameter-dependent discrete time Riccati equation 
(30) has a stabilizing solution X„ and U x = I - y^B^X^ > and p 2 I - HllI 2 H K > . 
Q. E. D. 

5. Numerical examples 

In this section, we present two examples to illustrate the design method given by Section 3 

and 4, respectively. 

Example 1 Consider the following discrete-time system in Peres and Geromel (1993) 

x(k + 1) = Ax(k) + BjWik) + B 2 u(k) 
z(k) = C 1 x(k) + D 12 u(k) 

where, 



0.2113 0.0087 0.4524 
0.0824 0.8096 0.8075 
0.7599 0.8474 0.4832 



0.6135 0.6538 
0.2749 0.4899 
0.8807 0.7741 



C a 



"1 0" 




"0 0" 


10 







1 


,D 12 = 










1 







1 



and B 1 = I 



In this example, we will design the above system under the influence of state feedback of the 
form (3) by using the discrete-times state feedback mixed LQR/ H^ control method 
displayed in Theorem 3.1. All results will be computed by using MATLAB. The above 
system is stabilizable and observable, and satisfies Assumption 3, and the eigenvalues of 
matrix A are pj = 1.6133, p 2 =0.3827 , p 3 = -0.4919 ;thus it is open-loop unstable. 



Let / = 2.89 , R -- 





[1 


1 






1 


>Q = 


10 




1 



we solve the discrete-time Riccati equation (13) to get 



X, 



2.9683 1.1296 0.1359 
1.1296 6.0983 2.4073 
0.1359 2.4073 4.4882 



>0. 
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U a =I-y- 2 BjXBi 



0.6446 -0.1352 -0.0163 
-0.1352 0.2698 -0.2882 
-0.0163 -0.2882 0.4626 



>0. 



Thus the discrete-time state feedback mixed LQR/ H m controller is 

K 



-0.3640 -0.5138 -0.3715 
-0.2363 -0.7176 -0.7217 



Example 2 Consider the following discrete- time system in Peres and Geromel (1993) 

x(k + 1) = Ax(k) + B 1 w(fc) + B 2 u(k) 
z(k)=C 1 x(k) + D 12 u(k) 

under the influences of state feedback with controller unceratinty of the form (4), where, A , 
Bj , B 2 , C 1 and D 12 are the same as ones in Example 1; the controller uncertainty AF(k) 
satisfies 

AF(k) = E K F(k)E K ,F T (k)F(k)<I 



where, E K ■■ 



"1 0" 




10 


,n K = 


11 


*■ 



0.0100 
0.0100 



In this example, we illustrate the proposed method by Theorem 4.1 by using MATLAB. As 
stated in example 1, the system is stabilizable and observable, and satisfies Assumption 3, 
and is open-loop unstable. 

"1 0" 

, p = 3.7800 , and S = 0.0010 , then we solve the 



Let y=8.27, R-- 



1 



1 



, Q 



"1 





0~ 





1 











1 



parameter-dependent discrete-time Riccati equation (21) to get 



X, 



18.5238 3.8295 0.1664 
3.8295 51.3212 23.3226 
0.1664 23.3226 22.7354 



>0, 



U 1 =J-y 2 ^'X B) B 1 



0.7292 -0.0560 -0.0024 
-0.0560 0.2496 -0.3410 
-0.0024 -0.3410 0.6676 



>0, U 7 



609.6441 723.0571 
723.0571 863.5683 



p 2 I-H T K U 2 H K - 



14.2274 -0.0723 

-0.0723 14.2020 

14.2884 



>0. 



176 New Trends in Technologies 

Based on this, the non-fragile discrete-time state feedback mixed LQR/ H x controller is 



-0.4453 -0.1789 -0.0682 
-0.1613 -1.1458 -1.0756 



6. Conclusion 

In this chapter, we first study the discrete time state feedback mixed LQR/ H x control 
problem. In order to solve this problem, we present an extension of the discrete time 
bounded real lemma. In terms of the stabilizing solution to a discrete time Riccati equation, 
we derive the simple approach to discrete time state feedback mixed LQR/ H x control 
problem by combining the Lyapunov method for proving the discrete time optimal LQR 
control problem with the above extension of the discrete time bounded real lemma, the 
argument of completion of squares of Furuta & Phoojaruenchanachi (1990) and standard 
inverse matrix manipulation of Souza & Xie (1992) .A related problem is the standard H^ 
control problem (Doyle et al., 1989a; Iglesias & Glover, 1991; Furuta & Phoojaruenchanachai, 
1990; Souza & Xie, 1992; Zhou et al. 1996), another related problem is the H^ optimal 
control problem arisen from Basar & Bernhard (1991). The relations among the two related 
problem and mixed LQR/ H m control problem can be clearly explained by based on the 
discrete time reference system (9)(3). The standard H„ control problem is to find an 
admissible controller K such that the H^ -norm of closed-loop transfer matrix from 
disturbance input w to the controlled output z is less than a given number y > while the 
H m optimal control roblem arisen from Basar & Bernhard (1991) is to find an admissible 
controller such that the H m -norm of closed-loop transfer matrix from disturbance input w 
to the controlled output z is less than a given number y > for the discre time reference 
system (9) (3). Since the latter is equivalent to the problem that is to find an admissible 
controller K such that sup^^ inf K {/} , we may recognize that the mixed LQR/ H M control 
problem is a combination of the standard H x control problem and H x optimal control 
problem arisen from Basar & Bernhard (1991). The second problem considered by this 
chapter is the non-fragile discrete-time state feedback mixed LQR/ H M control problem 
with controller uncertainty. This problem is to extend the results of discrete-time state 
feedback mixed LQR/ H„ control problem to the system (2)(4) with controller uncertainty. 
In terms of the stabilizing solution to a parameter-dependent discrete time Riccati equation, 
we give a design method of non-fragile discrete-time state feedback mixed LQR/ H x 
controller, and derive necessary and sufficient conditions for the existence of this non- 
fragile controller. 

7. References 

T. Basar, and Bernhard P. (1991). H m -optimal control and related minmax design problems: 

a dynamic game approach. Boston, MA: Birkhauser. 
D. S. Bernstein, and Haddad W. M. (1989). LQG control with an H w performance bound: A 

Riccati equation approach, IEEE Trans. Aut. Control. 34(3), pp. 293- 305. 
J. C. Doyle, Glover K., Khargonekar P. P. and Francis B. A. (1989a) . State-space solutions to 

standard H 2 and H ro control problems. IEEE Trans. Aut. Control, 34(8), pp. 831- 

847. 



Discrete Time Mixed LQR/H„ Control Problems 177 

J. C. Doyle, Zhou K., and Bodenheimer B. (1989b). Optimal control with mixed H 2 and H^ 

performance objective. Proceedings of 1989 American Control Conference, Pittsburh, 

PA, pp. 2065- 2070, 1989. 
J. C. Doyle, Zhou K., Glover K. and Bodenheimer B. (1994). Mixed H 2 and H„ perfor- 
mance objectives II: optimal control, IEEE Trans. Aut. Control, 39(8), pp.1575- 1587. 
D. Famularo, Dorato P., Abdallah C. T., Haddad W. M. and Jadbabaie A. (2000). Robust non- 
fragile LQ controllers: the static state case, INT. ]. Control, 73 (2),pp.l59-165. 
K. Furata, and Phoojaruenchanachai S. (1990). An algebraic approach to discrete- time H x 

control problems. Proceedings of 1990 American Control Conference, San Diego, pp. 

3067-3072,1990. 
W. M. Haddad, and Corrado J. R. (2000). Robust resilient dynamic controllers for systems 

with parametric uncertainty and controller gain variations, INT. ]. Control, 73(15), 

pp. 1405- 1423. 
P. A. Iglesias, and Glover K. (1991). State-space approach to discrete-time H^ control, INT. 

}. Control, 54(5), pp. 1031- 1073. 
L. H. Keel, and Bhattacharyya S. P. (1997). Robust, fragile, or optimal ? IEEE Trans. Aut. 

Control, 42(8), pp. 1098-1105 
L. H. Keel, and Bhattacharyya S. P. (1998). Authors' Reply. IEEE Trans. Aut. Control, 43(9), 

pp. 1268-1268. 
P. P. Khargonekar, and Rotea M. A. (1991). Mixed H 2 / H m control: A convex optimization 

approach, IEEE Trans. Aut. Control, 36(7), pp. 824-837. 
V. Kucera (1972). A Contribution to matrix quadratic equations. IEEE Trans. Aut. Control, 

17(3), pp. 344-347. 
D. J. N. Limebeer, Anderson B. D. O., Khargonekar P. P. and Green M. (1992). A game 

theoretic approach to H n control for time-varying systems. SIAM J. Control and 

Optimization, 30(2), pp.262-283. 
D. J. N. Limebeer, Anderson B. D. O., and Hendel B. (1994). A Nash game approach to 

mixed H 2 / H„ control. IEEE Trans. Aut. Control, 39(1), pp. 69-82. 
K. Ogata (1987). Discrete-time control systems. Prentice Hall, 1987. 
T. Pappas, Laub A. J., Sandell N. R., Jr. (1980). On the numerical solution of the discrete - 

time algebraic Riccati equation. IEEE Trans. Aut. Control, 25(4), pp. 631-641. 
P. L. D. Peres and Geromel J. C. (1993). H 2 control for discrete-time systems optimality and 

robustness. Automatica, Vol. 29, No. 1, pp. 225-228. 
J. E. Potter (1966). Matrix quadratic solution. /. SIAMApp. Math., 14, pp. 496-501. 
P. M. Makila (1998). Comments "Robust, Fragile, or Optimal ?". IEEE Trans. Aut. Control., 

43(9), pp. 1265-1267. 
M. A. Rotea, and Khargonekar P. P. (1991). H 2 -optimal control with an H^ -constraint: the 

state-feedback case. Automatica, 27(2), pp. 307-316. 
H. Rotstein, and Sznaier M. (1998). An exact solution to general four-block discrete-time 

mixed H 2 / H^ problems via convex optimization, IEEE Trans. Aut. Control, 43(10), 

pp. 1475-1480. 
C. E. de Souza and Xie L. (1992). On the discrete-time bounded real lemma with application 

in the characterization of static state feedback H^ controllers, Systems & Control 

Letters, 18, pp. 61-71. 



178 New Trends in Technologies 

M. Sznaier (1994). An exact solution to general SISO mixed H 2 / H x problems via convex 

optimization, IEEE Trans. Aut. Control, 39(12), pp. 2511-2517. 
M. Sznaier, Rotstein H. , Bu J. and Sideris A. (2000). An exact solution to continuous-time 

mixed H 2 / H^ control problems, IEEE Trans Aut. Control, 45(11), pp.2095-2101. 
X. Xu (1996). A study on robust control for discrete- time systems with uncertainty, A Master 

Thesis of 1995, Kobe university, Kobe, Japan, January,1996. 
X. Xu (2007). Non-fragile mixed LQR/ H x control problem for linear discrete- time systems 

with controller uncertainty. Proceedings of the 26th Chinese Control Conference. 

Zhangjiajie, Hunan, China, pp. 635-639, July 26-31, 2007. 
X. Xu (2008). Characterization of all static state feedback mixed LQR/ H x controllers for 

linear continuous-time systems. Proceedings of the 27th Chinese Control Conference. 

Kunming, Yunnan, China, pp. 678-682, July 16-18, 2008. 
G. H. Yang, Wang J. L. and Lin C. (2000). H x control for linear systems with additive 

controller gain variations, INT. }. Control, 73(16), pp. 1500-1506. 
G. H. Yang, Wang J. L. (2001). Non-fragile H„, control for linear systems with multiplicative 

controller gain variations, Automatica, 37, pp. 727-737. 
H. Yeh, Banda S. S. and Chang B. (1992). Necessary and sufficient conditions for mixed H 2 

and H m optimal control, IEEE Trans. Aut. Control, 37 (3), PP. 355-358. 
K. Zhou, Glover K., Bodenheimer B. and Doyle J. C. (1994). Mixed H 2 and H^ performance 

objectives I: robust performance analysis, IEEE Trans. Aut. Control, 39 (8), PP. 1564- 

1574. 
K. Zhou, Doyle J. C. and Glover K. (1996). Robust and optimal control, Prentice-Hall, INC., 

1996 



11 



Robust Control Design of Uncertain 
Discrete-Time Systems with Delays 

Jun Yoneyama, Yuzu Uchida and Shusaku Nishikawa 

Aoyama Gakuin University 
Japan 



1. Introduction 

When we consider control problems of physical systems, we often see time-delay in the 
process of control algorithms and the transmission of information. Time-delay often appear in 
many practical systems and mathematical formulations such as electrical system, mechanical 
system, biological system, and transportation system. Hence, a system with time-delay is a 
natural representation for them, and its analysis and synthesis are of theoretical and practical 
importance. In the past decades, research on continuous-time delay systems has been active. 
Difficulty that arises in continuous time-delay system is that it is infinite dimensional and a 
corresponding controller can be a memory feedback. This class of controllers may minimize 
a certain performance index, but it is difficult to implement it to practical systems due to 
a memory feedback. To overcome such a difficulty, a memoryless controller is used for 
time-delay systems. In the last decade, sufficient stability conditions have been given via 
linear matrix inequalities (LMIs), and stabilization methods by memoryless controllers have 
been investigated by many researchers. Since Li and de Souza considered robust stability 
and stabilization problems in (8), less conservative robust stability conditions for continuous 
time-delay systems have been obtained ((7), (11)). Recently, Hco disturbance attenuation 
conditions have also been given ((10), (15), (16)). 

On the other hand, research on discrete-time delay systems has not attracted as much attention 
as that of continuous-time delay systems. In addition, most results have focused on state 
feedback stabilization of discrete- time systems with time- varying delays. Only a few results 
on observer design of discrete-time systems with time-varying delays have appeared in the 
literatureffor example, (9)). The results in (3), (12), (14), (18) considered discrete-time systems 
with time-invariant delays. Gao and Chen (4), Hara and Yoneyama (5), (6) gave robust 
stability conditions. Fridman and Shaked (1) solved a guaranteed cost control problem. 
Fridman and Shaked (2), Yoneyama (17), Zhang and Han (19) considered the Hoo disturbance 
attenuation. They have given sufficient conditions via LMIs for corresponding control 
problems. Nonetheless, their conditions still show the conservatism. Hara and Yoneyama 
(5) and Yoneyama (17) gave least conservative conditions but their conditions require many 
LMI slack variables, which in turn require a large amount of computations. Furthermore, 
to authors' best knowledge, few results on robust observer design problem for uncertain 
discrete-time systems with time-varying delays have given in the literature. 
In this paper, we consider the stabilization for a nominal discrete-time system with 
time-varying delay and robust stabilization for uncertain system counterpart. The system 
under consideration has time-varying delays in state, control input and output measurement. 
First, we obtain a stability condition for a nominal time-delay system. To this end, we define 
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a Lyapunov function and use Leibniz-Newton formula and free weighting matrix method. 
These methods are known to reduce the conservatism in our stability condition, which are 
expressed as linear matrix inequality. Based on such a stability condition, a state feedback 
design method is proposed. Then, we extend our stabilization result to robust stabilization for 
uncertain discrete-time systems with time-varying delay. Next, we consider observer design 
and robust observer design. Similar to a stability condition, we obtain a condition such that 
the error system, which comes from the original system and its observer, is asymptotically 
stable. Using a stability condition of the error system, we proposed an observer design 
method. Furthermore, we give a robust observer design method for an uncertain time-delay 
system. Finally, we give some numerical examples to illustrate our results and to compare 
with existing results. 

2. Time-delay systems 

Consider the following discrete-time system with a time-varying delay and uncertainties in 
the state and control input. 

x(k + 1) = (A + AA)x(k) + (A d + AA d )x(k - d k ) + (B + AB)u(fc) 

+ (B d + AB d )u(k-d k ) (1) 

where x(k) 6 5R" is the state and u (k) g 5R m is the control. A, A d , B and B d are system matrices 
with appropriate dimensions. d k is a time-varying delay and satisfies < d m < d k < d^ and 
d k+ \ < d k where d m and d^[ are known constants. Uncertain matrices are of the form 

[AA AA d AB AB d ] = HF(k) [E E d E l E b ] (2) 

where F(k) 6 $v x 1 is an unknown time-varying matrix satisfying F T (k)F(k) < I and H, E, E d , 
E\ and E;, are constant matrices of appropriate dimensions. 

Definition 2.1. The system (1) is said to be robustly stable if it is asymptotically stable for all 
admissible uncertainties (2). 

When we discuss a nominal system, we consider the following system. 

x(k + l) = Ax{k)+A d x{k-d k ) + Bu{k) + B d u(k-d k ). (3) 

Our problem is to find a control law which makes the system (1) or (3) robustly stable. Let us 
now consider the following memoryless feedback: 

u(k) = Kx{k) (4) 

where K is a control gain to be determined. Applying the control (4) to the system (1), we have 
the closed-loop system 

x{k + l) = {{A + AA) + (B + AB)K)x{k) + ((A d + AA d ) + (B d + AB d )K)x{k-d k ). (5) 

For the nominal case, we have 

x(k + 1) = (A + BK)x(k) + (A d + B d K)x{k - d k ). (6) 

In the following section, we consider the robust stability of the closed-loop system (5) and the 

stability of the closed-loop system (6). 

The following lemma is useful to prove our results. 
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Lemma 2.2. ((13)) Given matrices Q = Q T , H, E and R = R T > with appropriate dimensions. 

Q + HF(k)E + E T F T (k)H T < 
for all F(k) satisfying F T (k)F(k) < R if and only if there exists a scalar e > such that 



Q 



l 



]T 



HH 1 +eE l RE < 



T r . 



3. Stability analysis 

This section analyzes the stability and robust stability of discrete-time delay systems. 
Section 3.1 gives a stability condition for nominal systems and Section 3.2 extends the stability 
result to a case of robust stability. 

3.1 Stability for nominal systems 

Stability conditions for discrete-time delay system (6) are given in the following theorem. 

Theorem 3.1. Given integers d m and d^, and control gain K. Then, the time-delay system (6) is 
asymptotically stable if there exist matrices Pi > 0, P2 > 0, Qi > 0, Q2 > 0, S > 0, M > 0, 



satisfying 



where 



<P 



riii 




■Nil 




Til 


L 2 




N 2 




T 2 


13 


, N = 


N 3 


, T = 


r 3 


14 




Ni 




r 4 


U 5 J 




-N 5 \ 




-T5J 



#i + H L 


+ sl 




\l2 
S 


*i = 


"Pi 0" 
*22 
-Qi-M 
$44 -P 2 

_0 -P2P2-Q2- 


' 


<*>22 = - 


-P\ + Qi + (d M - dm + 1)M, 


<E>44 = I 


'2 + Q2 + ^mS, 


Z = 


" " 



-Pi 
- Pi J 


+ N, 


Sl=[ 


L-L0-L0], 


s N =[ 


0N-N00], 


Sr=[ 


T -T 


A + BJC) -T(A d + B d K)0 


0] 



<0 



(7) 
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Proof: First, we note from Leibniz-Newton formula that 

2Z T (k)L[x(k + l)-x(k)-e(k)}=0, (8) 

2£ T (k)N[x(k)-x(k-d k )- £ e(i)]=0 (9) 

i=k-d t 
where e(k) = x(k + 1) — x(k) and 

f(k) = [x T (k + l) x T (k) x T (k-d k ) e T (k) e T (k-d k )]. 

It is also true that 

2?{k)T[x{k + 1) - (A + BK)x(k) - (A d + B d K)x(k - d k )\ = 0. (10) 

Now, we consider a Lyapunov function 

v{k) = Vi(k) + v 2 (k) + v 3 (k) + v 4 (k) 

where 

V 1 (k) = x T (k)P 1 x(k) + £ e T (i)P 2 £ e(i), 

i=k—d k i=k—di < 

V 2 (k)= £ x r (f)Qix(0+ E e T (f)Q 2 e(i), 

i=k—dfc i=k—d k 

v 3 (k) = E E « r (/)s«(/), 

i=—dk j=k+i 

W*W = E E x T (i)Mx(i), 

j=-d M i=k+j 

and Pj, P 2 , Q\, Q 2 , S and M are positive definite matrices to be determined. Then, we calculate 
the difference AV = V(k + 1) - V(k) and add the left-hand-side of equations (8)-(10). 
Since AVj(fc), i = 1, ■ ■ ■ ,4 are calculated as follows; 

k k 

AV 1 (k) = x T (k + l)P 1 x(k + l)+ E e T (*')ft E e « 

i=k+\— dic+i i=k+\-d k+ i 

-x T {k)P l x{k)- E e T (i)P 2 £ e(») 

i=k—d k i=k—d k 

<x T {k + l)P lX (k + 1) - x T (fc)P!x(fc) + e T (k)P 2 e{k) 
-2e T (k)P 2 e(k-d k )+2e T (k)P 2 £ e(i) 

i=k—d k 

k-\ 

+e T (k-d k )P 2 e(k-d k )-2e T (k-d k )P 2 £ e(i), 

i=k—di 
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AV 2 (k)= £ x T (i)Q lX (i)+ £ e T (i)Q 2 e(i) 

i=k+l-d k+1 i=k+l-d k+1 

£ x T (0Qi*(0-_E e T (i)Q 2 e(i) 

< x T \k)Q lX (k) + e T (k)Q 2 e(k) - x T (k - d k )Q x x{k - d k ) 
-e T (k-d k )Q 2 e(k-d k ), 

k-l k-l 



AV 3 (k) = d k+l e l (k)Se(k) - £ e 1 (i)Se( 



E e l (i)Se{i 



i=k—d k 



i=k-d k+1 
k-l 

< d M e T (k)Se{k) - £ e T (i)Se{i), 

i—k—d k 

k-d m 

AV A (k) = (d M - d m + l)x T (k)Mx(k) - £ x T (i)Mx(i) 

i=k-d M +l 

< (d M - d m + l)x T (k)Mx(k) - x T (k - d k )Mx(k - d k ), 
we have 

AV(fc) = AVx(fc) + AV 2 (k) + AV 3 {k) + AV 4 (fc) 

<£ r (fc)[4> 1 + S L + E[ + E N + s£ + S T + E r ]£(/c) + £ ? r (fc)ZS- 1 Z T f(fc) 

i=k-d k 

- £ (f T (fc)z + e r (OS)s- 1 (z r f(fc) + s e (0) 

< f T (fc)[O a + E L + s£ + E N + e£ + S T + e£ + d M ZS- 1 Z T ]^(fc) 



If (7) is satisfied, by Schur complement formula, we have <t>i + Ej, + E^ + Ejv + S^ + Sy + 
Ep + iiMZS _1 Z T < 0. It follows that AV{k) < and hence the proof is completed. 

k-l k-l 

Remark 3.2. We employ VJ {*) in our Lyapunov function instead of VJ (*)• This gives a less 

i—k—d k i=k—d M 

conservative stability condition. 

3.2 Robust stability for uncertain systems 

By extending Theorem 3.1, we obtain a condition for robust stability of uncertain system (5). 

Theorem 3.3. Given integers d m and d^, and control gain K. Then, the time-delay system (5) is 
robustly stable if there exist matrices Pi > 0, P 2 > 0, Qi > 0, Q 2 > 0, S > 0, M > 0, 
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and a scalar A > satisfying 



II 



<I> + AE r E H T 
H -XI 



<0, 



(11) 



where <J> is given in Theorem 3.1, and 



H 



t t t 



-WT 



-H T TT -H t T? -H T Tj -H r T 5 T 0] 



and 



E + ExK E d + E b ]C 



Proof: Replacing A, A A , B and B d in (7) with A + HF{k)E, A d + HF(k)E d , B + HF(k)E l and 
B + HF(k)Ei,, respectively, we obtain a robust stability condition for the system (5). 



$ + H 1 F(k)E + E 1 F 1 (k)H < 



(12) 



By Lemma 2.2, a necessary and sufficient condition that guarantees (12) is that there exists a 
scalar A > such that 



<t> + A£ T E 



H T H < 



(13) 



Applying Schur complement formula, we can show that (13) is equivalent to (11). 

4. State feedback sabilization 

This section proposes a state feedback stabilization method for the uncertain discrete-time 
delay system (1). First, stabilization of nominal system is considered in Section 4.1. Then, 
robust stabilization is proposed in Section 4.2 



4.1 Stabilization 

First, we consider stabilization for the nominal system (3). Our problem is to find a 
control gain K such that the closed-loop system (6) is asymptotically stable. Unfortunately, 
Theorem 3.1 does not give LMI conditions to find K. Hence, we must look for another method. 

Theorem 4.1. Given integers d m and d^, and scalars p ir i = 1, • • • , 5. Then, the controller (4) 
asymptotically stabilizes the time-delay system (3) if there exist matrices Pj > 0, Pi > 0, Qi > 0, 
Q 2 > 0, S > 0, M > 0, G, y 
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satisfying 
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Y a + e L + ®l + & N + e 



td M Z 
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(14) 
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where 
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Y44 
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-P 2 P 2 


-Qi\ 
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-Pi + Ql + (<*M 

P 2 + Q2 + d M S, 




-h 

P2 
[L -L -I 0] , 

[0N-N00], 

p{Y T - Pl {AY T + BG) 
p 2 Y T -p 2 (AY T + BG) 
P3 Y T -p 3 (AY T + BG) 
p 4 Y T -p A {AY T + BG) 
,p 5 Y T -p 5 (AY T + BG) 



1)M, 



- Pl (A d Y T + B d G)0O 
-p 2 (A d Y T + B d G) 
-p 3 (A rf Y r + B rf G) 
-p 4 (A d Y r + B d G)0 
-p 5 (^V T + B d G)0 0. 



In this case, a controller gain in the controller (4) is given by 



K=GY 



-T 



(15) 



PiV- 



1, ■ 



,5 where each p, is given. We substitute them into (7). Then, 
diag[Y YYYY]. Defining P, = YPiY T , Q, = YQjY T , i = 



Proof: Let I, 

we calculate W = E<I>E r with E 

1,2, S = YSy T , M = YMY r , L = YLY T , N = YNY T , we obtain < in (14) where we let 

G = KY . If the condition (14) hold, state feedback control gain matrix K is obviously given 

by (15). 

Remark 4.2. Should Y be singular, Let Lj = 0. In this case, it follows from (1, l)-block ofW that 
Pi + Pi(^ + ^ T ) < 0. Then, if (14) holds, Y must be nonsingular. 

4.2 Robust stabilization 

In a similar way to robust stability, we extend a stabilization result in the previous section to 
robust stabilization for uncertain discrete-time delay system (1). 



Theorem 4.3. Given integers d m and d^, and scalars p,, i = 1, 
robustly stabilizes the time-delay system (1) if there exist matrices P\ 
S > 0, M > 0, G, Y 



■ ■ ,5. Then, the controller (4) 
>0,P 2 > 0, Qj > 0, Q 2 > 0, 
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and a scalar A > satisfying 



A 



W + AH T H E T 
E -\l 



<0, 



(16) 



where 



and 



H 



[- Pl H T -p 2 H T -p 3 H T -p 4 H T -p 5 H T 0] 



E = [0 EY T + E X G E d Y T + E b G 0] . 
In this case, a controller gain in the controller (4) is given by (15). 

Proof: Replacing A, A d , B and B d in (14) with A + HF(k)E, A d + HF(k)E d , B + HF(A:)£ 1 and 
B + HF(k)Et„ respectively, we obtain robust stability conditions for the system (1): 



Y + H 1 F{k)E + E 1 F' (k)H < 



(17) 



By Lemma 2.2, a necessary and sufficient condition that guarantees (17) is that there exists a 
scalar A > such that 



W + AH T H 



E T E<0 



(18) 



Applying Schur complement formula, we can show that (18) is equivalent to (16). 

5. State estimation 

All the information on the state variables of the system is not always available in a physical 
situation. In this case, we need to estimate the values of the state variables from all the 
available information on the output and input. In the following, we make analysis of the 
existence of observers. Section 5.1 analyzes the observer of a nominal system, and Section 5.2 
considers the robust observer analysis of an uncertain system. 

5.1 Observer analysis 

Using the results in the previous sections, we consider an observer design for the system (1), 
which estimates the state variables of the system using measurement outputs. 



x(k + 1) = ( A + AA)x(k) + (A d + AA d )x(k - d k ), 
y(k) = (C + AC)x(fc) + (Q + AQ)x(fc - d k ) 



(19) 
(20) 



where uncertain matrices are of the form: 

'AA AA d 
AC AQ, 



H 
H 2 



F(k) [E E d ] 



where F(k) G W x l is an unknown time-varying matrix satisfying F T (k)F(k) < I and H, H2, E 

and E d are constant matrices of appropriate dimensions. 

We consider the following system to estimate the state variables: 



x(k+l) = Ax{k) + K(y(k) - Cx{k)) 



(21) 
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where x is the estimated state and K is an observer gain to be determined. It follows from (19), 
(20) and (21) that 



x c (k + 1) = (A + HF(k)E)x c (k) + (A d + HF(k)E d )x c (k - d k ). 
where xj = [x e T ] T , e(k) = x(k) — x(k) and 



(22) 



A 
A-KC 



A d 
A d - KC d 



H 



,E=[E0],E d = [E d 0] 



H 
H-KH 2 

We shall find conditions for (22) to be robustly stable. In this case, the system (21) becomes an 
observer for the system (19) and (20). 
For nominal case, we have 



x c (k + l) = Ax c (k) + A d x c {k-d k ). 



(23) 



We first consider the asymptotic stability of the system (23). The following theorem gives 
conditions for the system (23) to be asymptotically stable. 

Theorem 5.1. Given integers d m and d^, and observer gain K. Then, the system (23) is 
asymptotically stable if there exist matrices < P a G K 2 " x2 ", < P 2 G SR 2nx2 », < Qi G K 2nx2 ", 
< Q 2 G sR 2nx2n , < 5 G K 2nx2 ", < M G 5R 2nx2n , 



G 5R 



10nx2« 



, N 



N 2 
N 3 

N 5 



G 5R 



Wnxln 



satisfying 



where 



<t> 



-sf 


s n + Zj] + Er 


+ 


ST 






d M Z J 










Pi 





















$22 





















-Qi- 


M 





















$44 




-Pi 



ti 
f 2 
f 3 
% 
%. 

iduZ 
-S 



G 5R 



Wnxln 



<0 



(24) 



*1 



--P2P2-Q2 

$22 = -Pi + Qi + (d M - d m + 1)M, 
$44 = ^2 + Q2 + dMS, 



+N, 
-Pi 
Pi . 
H L =[L-L0 -L 0] , 

E N = [0N-N0O], 

H r = [f -TA -TA d 0] . 
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Proof: We follow similar lines of proof of Theorem 3.1 for the stability of the system (23). Then, 
the result is straightforward. 

5.2 Robust observer analysis 

Now, we extend the result for the uncertain system (23). 

Theorem 5.2. Given integers d m and dyi, and observer gain K. Then, the system (22) is robustly stable 
if there exist matrices < P\ G 5R 2nx2 ", < P 2 G $l 2nx2n , < Qi G K 2nx2 ", < Q 2 G 5R 2nx2n , 
0< S G 5R 2 " x2 ",0<MG K 2nx2n , 



G 5R 



10nx2n 



N 



N 2 
N 3 
N 4 
N 5 



G 3? 



10nx2n 



G 3? 



10nx2n 



and a scflkr A > satisfying 



II 



<I> + A£ T £ H T 
H -A7 



<0 



ryftere <I> t's given in Theorem 5.1, and 

H 

E = [0 £ £ rf 000 



-H r f 1 r -H T TJ -H T f 3 T -H r T 4 T -H T f 5 r 0] 



Proof: Replacing A and A d in (24) with A + HF(k)E and A d + HF(k)E d , respectively, and 
following similar lines of proof of Theorem 3.3, we have the desired result. 

6. Observer design 

This section gives observer design methods for discrete-time delay systems. Section 6.1 
provides an observer design method for a nominal delay system, and Section 6.2 proposes 
for an uncertain delay system. 



6.1 Nominal observer 

Similar to Theorem 3.1, Theorem 5.1 does not give a design method of finding an observer 
gain K. Hence, we obtain another theorem below. 

Theorem 6.1. Given integers d m and d^, and scalars p,- and p,, i = 1, • • • ,5. Then, (21) becomes 
an observer for the system (19) and (20) with A A = AA d = 0, AC = AQ = if there exist matrices 
< Pi G 5R 2nx2n , < P? G 5R 2nx2 ", < Qi G SR 2nx2n n ^ r\. a s»2nx2n n ^ c ^ 8r>2nx2n 
< M G SR 2 " x2 ",'G G K nx ", Y G 5R"' X " 



G 5R 



10nx2n 



N 



N 2 
N 3 
N 4 
#5. 



G !R 



10»x2n 






< Q 2 G SR 2nx2n , 


< 


: S G 5R 2 " 


Til 




"fll 




T? 




T 2 




T 3 


G SR 5 " x »,f = 


?3 


G 3? 5nx " 


T 4 




T 4 




T 5 J 




A 
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satisfying 



where 



¥ 



% + Q L + &l + & N + Ql + Q T + © 

vcImZ 



AmZ 



-s 



<0 



(25) 



"Pi 

f 22 

-Qi- 





-Pi + Qi + (d M 

*44 =Pl + Q2 + d M §, 

r o 



+N, 

-Pi 

L P 2 

I -to -to], 

"0 N -N 0] , 
Ti piY -TiA - 
fj pjY -f x A - 

T 2 p 2 Y -T 2 A - 
T 2 p 2 Y -T 2 A - 
T 3 p 3 Y -T 3 A - 

% P$Y ~%A - 
T 4 p 4 Y -T 4 A - 
f 4 p 4 Y -T A A - 
T 5 p 5 Y -T 5 A - 
lt 5 p 5 Y -t 5 A - 



*1 = 



f 2 2 = 7 , 



z = 



©L = 

&N = 






M 
Y 44 

-?2 

- d m + 








-h 

P2-Q2 

1)M, 



T = 



Pi(YA 
pliYA 
pi{YA 
P3(YA 
MYA 
Pi(YA 
MYA 
P5(YA 
P5(YA 



- GC) -T r A d 

- GC) -TiA d 

- GC) -T 2 A d 

- GC) -t 2 A d 

- GC) -T 3 A d 

- GC) -t 3 A d 

- GC) -T 4 A d 

- GC) -t 4 A d 

- GC) -T 5 A d 

- GC) -%A d 



Pi(YA d 
Pi(YA d 
Pi(YA d 
Pi(YA d 
p3(YA d 
p3(YA d 
p4(YA d 
Pi(YA d 
P5(YA d 
P5(YA d 



GC d ) 0- 
GC d ) 
GC d ) 
GC d ) 
GC d ) 
GC d ) 
GC d ) 
GC d ) 
GC d ) 
GQ) 0. 



In this case, an observer gain in the observer (21) is given by 



K = Y~ l G. 



Proof: Proof is similar to that of Theorem 4.1. Let 



(26) 



Ti 



T, PiY 
% PiY. 



!,••• ,5 



where p- t and p u i = 1, ■ ■ ■ ,5 are given. We substitute them into (24). Defining G = YK, we 
obtain T < in (25). If the condition (25) hold, observer gain matrix K is obviously given by 
(26). 



6.2 Robust observer 

Extending Theorem 4.1, we have the following theorem, which proposes a robust observer 
design for an uncertain delay system. 
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Theorem 6.2. Given integers d m and dyi, and scalars p±, pi, i = 1, ■ ■ ■ , 5. Then, (21) becomes an 
observer for the system (19) and (20) if there exist matrices < Pj G 5R 2 " x2 ", < P 2 G K 2 " x2 ", 
< Qj G 5ft 2 " x2tt , < Q 2 G K 2nx2tt , < S G 5ft 2 " x2n , < M G 5R 2 " x2n , G G » n>< ", Y G 5R" X " 



\10nx2n 



, N 



and a scflkr A > satisfying 



No 
N 3 
N 4 
N 5 



A 



G 5R 



10nx2n 



, r 



G 5R 



5j; xh 



, f 



G K ; 



5nxn 



Y + AE T E H T 
H -M 



<0 



(27) 



a^ere "T is given in Theorem 6.1, and 



H 



H l T{ + Pl {YH-GH 2 )' H l T[ + pi(YH - GH 2 ) 1 H l T{ + p 2 (YH - GH 2 ) 



H T TJ + p 2 (YH - GH 2 ) T H T T? 
H T Tj + Pi (YH - GH 2 ) T H T fJ 
H r f 5 T + p 5 (YH - GH 2 ) T 0] , 
[00 EO E rf 00 00000] . 



p 3 (YH-GH 2 ) T H T n 
p 4 (YH^GH 2 ) T H T Tj 



p 3 (YH - GH 2 ) T 
p 5 (YH-GH 2 ) T 



In this case, an observer gain in the observer (21) is given by (26). 

Proof: Replacing A and A d in (27) with A + HF(k)E and A d + HF(k)E d , respectively, and 
following similar lines of proof of Theorem 4.3, we have the desired result. 



7. Examples 

In this section, the following examples are provided to illustrate the proposed results. First 
example shows stabilization and robust stabilization. Second one gives observer design and 
robust observer design. 

Example 7.1. Consider the following discrete-time delay system: 



x(k + l) 



+ 



1.1 + a 
0.97 
0.1 



0.5 



u{k) + 



x(k)- 

0.2 

0.3 



-0.1 
-0.1 

u(k-d k ) 




-0.1 



x(k-d k ) 



where a satisfies \tx\ < a for a. is an upper bound of a(k). First, we consider the stabilization for a 
nominal time-delay system with a(k) =0 by Theorem 4.1. Table 1 shows control gains for different 
time-invariant delay d k , while Table 2 gives control gains for different time-varying delay d k . 
Next, we consider the robust stabilization for the uncertain time-delay system with a(k) ^ 0. In this 
case, system matrices can be represented in the form of(l) with matrices given by 



1.1 
0.97 



-0.1 
-0.1 -0.1 



[10] 



[00] 



0.1 
0.5 



0.2 
0.3 



H 



F(k) 



«(fc) 



Robust Control Design of Uncertain Discrete-Time Systems with Delays 



191 





The stabilizat 


h 


p's 


K 




l 


[0.1 0.1 - 0.1 0.5 0.1] 


[-1.1316 -0 


1360] 




2 


[0.1 0.1 - 0.1 0.5 0.1] 


[-0.9690 -0 


0976] 




3 


[0.1 0.1 -0.1 0.5 0.1] 


[-0.7908 -0 


0545] 




4 


[0.09 0.05 -0.1 0.55 0.1] 


[-0.5815 -0 


0306] 


Table 1. 


ion for time-invariant delay d k 






d k 


p's 


K 




0<d k <l 


[0.1 0.1 -0.1 0.5 0.1] 


[-1.1209 


- 0.1174] 




0<d k <2 


[0.1 0.1 -0.1 0.5 0.1] 


[-0.9429 


- 0.0839] 




0<d k <3 


[0.1 0.1 -0.1 0.5 0.1] 


[-0.7950 


- 0.0469] 




0<d k <4 


[0.09 0.05 - 0.1 0.55 0.1] 


[-0.5586 


- 0.0253] 


Table 2. 


The stabilization for time-varying delay d k 






d k 


a 


p's 


K 




3 


0.05 


[0.1 0.1 - 0.1 0.5 0.1] 


[-0.8622 - 


- 0.0059] 




3 


0.10 


[0.1 0.1 -0.1 0.5 0.1] 


[-0.6243 - 


- 0.0000] 




2 


0.15 


[0.12 0.12 - 0.1 0.5 0.05] 


[-1.2515 - 


- 0.0115] 


Table 3. 


The robust stabilization for time-invariant delay d k 






d k 


a. 


p's 


K 




< 4 < 3 


0.05 


[0.1 0.1 - 0.1 0.5 0.1] 


[-0.8394 -0.0047] 




< d k < 3 


0.10 


[0.12 0.1 -0.1 0.5 0.1] 


-1.2539 -0.0108] 




< 4 < 2 


0.15 


[0.12 0.12 - 0.1 0.5 0.05] 


-1.1740 -0.0015] 



Table 4. The robust stabilization for time-varying delay d k 

For time-invariant delay d k , Theorem 4.3 gives control gains for different a in Table 3. Table 4 provides 
the result for time-varying delay d k . 



Example 7.2. Consider the following discrete-time delay system: 



x(k + l) 



0.85 + 0.1a 

0.97 

0.5 0.2]x{k) 



x(fc)-f 

[0.1 0.1 " 



0.1 
-0.1 -0.1 

x{k-d k ) 



x{k - d k ), 



where a satisfies \tx\ < & for a. is an upper bound of a(k). We first consider the observer design for a 
nominal time-delay system with a{k) =0 by Theorem 6.1. Table 5 shows observer gains for different 
time-invariant delay d k , while Table 6 gives observer gains for different time-varying delay d k . In the 
following observer design, all p's are set to be zero for simplicity. 

Next, we consider the robust observer design for the uncertain time-delay system with a(k) ^ 0. In 
this case, system matrices can be represented in the form of (1) with matrices given by 



0.85 
0.97 



,A d 



0.1 
-0.1 




-0.1 



E= [ft0],E rf = £ 1 = [0 0] 



C= [0.5 0.2], Q= [0.1 0.1], H 



0.1 





, F(k) = <*). 
a 
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d-k 


p's 


K 


1 


[-22.6 2.5 - 2.1 - 1.9 - 1.9] 


0.4835 
0.3622 




2 


[-21.9 6.7 -0.3 -2.6 -1.9] 


0.5372 
0.3348 




3 


[-23.4 8.6 0.2 - 1.9 - 1.9] 


0.5174 
0.3910 




4 


[-9.0 26.4 0.2 - 2.5 - 1.9] 


-3.2281 
0.0459 




5 


[-9.0 25.9 0.2 - 2.5 - 1.9] 


-2.3508 
-0.7232 





Table 5. The observer design for time-invariant delay d k 



d k 


p's 


K 


0<d k <l 


[-20.2 6.5 -2.1 -2.5 -1.9] 


0.6295 
0.3777 




0<d k <2 


[-21.9 6.7 -0.3 -2.6 -1.9] 


0.5817 
0.3475 




0<d k <3 


[-22.0 8.6 0.2 - 1.9 - 1.9] 


0.5490 
0.3037 




< 4 < 4 


[-22.0 8.6 0.2 - 2.5 - 1.9] 


0.5157 
0.2921 




0<d k <5 


[-22.5 8.6 0.2 -1.9 -3.1] 


0.5170 
0.2956 





Table 6. The observer design for time-varying delay d k 



dk 


a. 


p's 


K 


1 


0.5 


[-22.6 2.5 -2.1 -1.9 -1.9] 


0.5047 
0.3813 




2 


0.4 


[-21.9 6.7 -0.3 -2.6 -1.9] 


0.5625 
0.3633 




3 


0.3 


[-23.4 8.6 0.2 - 1.9 - 1.9] 


0.5264 
0.3641 




4 


0.3 


[-9.0 26.4 0.2 - 2.5 - 1.9] 


-2.6343 
-1.6768 




5 


0.2 


[-9.0 25.9 0.2 - 2.5 - 1.9] 


-2.5959 
-1.5602 





Table 7. The observer design for time-invariant delay d k 

For time-invariant delay d k , Theorem 6.2 gives observer gains for different a in Table 7. Table 8 provides 
observer gains for time-varying delay d k by the same theorem. 



8. Conclusions 

In this paper, we proposed stabilization and robust stabilization method for discrete-time 
systems with time-varying delay. Our conditions were obtained by introducing new 
Lyapunov function and using Leibniz-Newton formula and free weighting matrix method. 
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d k 


a. 


p's 


K 


< d k < 1 


0.5 


[-20.2 6.5 - 2.1 - 2.5 - 1.9] 


0.6345 
0.3520 




< d k < 2 


0.4 


[-21.9 6.7 -0.3 -2.6 -1.9] 


0.5870 
0.3153 




< d k < 3 


0.3 


[-22.0 8.6 0.2 -1.9 -1.9] 


0.5675 
0.3003 




< d k < 4 


0.3 


[-22.0 8.6 0.2 -2.5 -1.9] 


0.5375 
0.3444 




< d k < 5 


0.2 


[-22.5 8.6 0.2 -1.9 -3.1] 


0.5077 
0.3425 





Table 8. The observer design for time-varying delay d k 

Similarly, we also gave observer design and robust observer design methods. Numerical 
examples were given to illustrate our proposed design method. 
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1. Introduction 

In the last twenty some years, much attention has been paid to the problem of fault-tolerant 
control in satellite attitude control systems and many methods have been developed and 
proven to be capable of tolerating certain types of system faults (see e.g., [1~5] and the 
references therein). However, these solutions focused mainly on keeping stability of the faulty 
systems and in less consideration of other performance indices. Actually, the performance 
requirements of practical satellite control systems are usually multi-objective even in faulty 
cases and it is desirable for fault tolerant systems to keep the required performance indices in a 
satisfactory and admissible region rather than the optimization of single index [6-7]. 
As is well known, in many practical applications, it is desirable to construct systems to 
achieve better transient property, strong anti-disturbance ability and adequate level of cost 
function performance. To this end, optimal controllers have been designed by assigning pole 
in a desired region (see e.g., [8] and [9]), using Hoc norm-bound constraint on disturbance 
attenuation [10, 11] and the guaranteed cost control (see [12] and [13]), respectively. 
Unfortunately, few results have been considered such performance indices simultaneously. 
Meanwhile, once some components of satellite attitude control systems go wrong, it is 
difficult to confirm desired multiple performances by the existing fault-tolerant control. 
Thus, it is necessary to investigate the problem of fault-tolerant control with multiple 
performance constraints. 

Therefore, it is our motivation to investigate the quadratic D stabilizable satisfactory fault- 
tolerant control problem with consistent indices constraints for a class of satellite attitude 
control uncertain discrete-time systems subject to actuator failures. In view of possible 
actuator failure as well as uncertainties which do not satisfy matching conditions existing in 
both the state and control input matrices, we first derive the existence conditions of 
satisfactory fault-tolerant state-feedback control law. Then by LMI technique, a convex 
optimization problem is formulated to find the corresponding controller. The state-feedback 
controller is designed to guarantee the closed-loop system satisfying the pre-specified 
quadratic D stabilizability index, H w norm-bound constraint on disturbance attenuation and 
having the quadratic cost performance simultaneously, for all admissible value-bounded 
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uncertainties and possible actuator failures. Furthermore, the consistency of the 
performance indices mentioned earlier is discussed for fault-tolerant control. Finally, 
simulative example is provided to illustrate the validity of the proposed method and the 
necessity of such a satisfactory fault-tolerant control. 

2. Problem formulation 

The systems considered in this paper can be described as follows: 

x(k + l) = (A + AA)x(k) + (B + AB)u f (k) + Va(k) 
z(k) = Cx(k) + Eu f (k) 

where x(k) e SR" is the state vector, u' (K)gW is the control input from the actuator that 
may be fault, z(k) e 9? is the controlled output, to(A:)e s JJ' ? is the input disturbance and 
| | a(k) | \ 2 < P , A, B, C and D are known real constant matrices with appropriate 
dimensions, AA and AB are unknown matrices representing parameter uncertainties in the 
state matrix and input matrix, respectively. 

In much literature, time-varying matrices of uncertain parameters are assumed to be of the 
form [AA AB] = HF(fc)[E a E;,], where H, E and E b are known real constant matrices 
with appropriate dimensions, F(lc) is an unknown real matrix satisfying 
F(fc) e CI :={F(lc) e SR' ' \ T T (k)T(k) <I) . However, the uncertainty is often value bounded 
which is more universal and need not satisfy the so-called matching conditions in practical 
engineering. Thus, AA and AB denote value bounded uncertainties in this paper, i.e., 
| | AA | |< a , | | AB | |< b . 
Suppose the states are available for state-feedback. 

«(*) = Kx(k) (2) 

where KsSR''" is the feedback gain matrix. For the control input, the following failure 
model in [18] is adopted for this study: 

u f (k) = Mu(k) (3) 

M = diag \m 1 ,m 2 , ■■■,m \ (4) 

where M denotes the actuator faults function matrix, < m i( < m, < m iu , m a < 1 , m iu > 1 , 
i = l,2,—,p. 

Remark 1: In the above fault matrix M , if m ; = 1 , it corresponds to the normal case 
vl* (k) = u(k) . If m i = , outage of actuator control signal occurs. If < m a < m t < m iu , m a < 1 , 
nt iM > 1 and m 1 ■ ^ 1 , the corresponding actuator would be in partial failure case. Hence, let 
u * (k) denote the control input vector both in normal and actuator failures cases for fault- 
tolerant control research in this paper. 

The decomposition of fault function M is given below with a similar manner in [7], which 
will be used for our main results. Define 

M = diag[m 01/ m 02 ,---,m 0p ] , J = diag[j 1 ,j 2 ,---,j p ], \^\= diag[\l 1 \ / \l 2 \ / --- / \l p \], 
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where m 0l = (m a + m m )/2, j { = (m iu - m a ) / (m iu + m a ) , l t = (m, - m 0i ) / m oi . So, we then have 

M = M (I + L) |L|<J<I (5) 

The faulty closed-loop system is given by 

x(fc + l) = A c x(/c) + Dco(fc) 
z(k) = C c x(k) 

where A c = A c + AA C , A c = A + BMK , AA C = AA + ABMK , C c = C + EMK . 

The cost function associated with system (1) considered possible actuator faults (3) is 

GO , 

/ = I> r W Q + (MK) T R(MK) x(fc) (7) 

k=o x ' 

where Q = Q >0, R=R >0 are given weighting matrices. 

Definition 1: For system (1), if there exists state-feedback controller, such that the faulty 

closed-loop system (6) will meet the following indices constraints simultaneously, 

a. The closed-loop system is quadratic D stabilizable with constraint <b(q,r) , (£>(q,r) 
denotes the disc with centre q + jO and the radius r , where r and q are known 
constants with | q \ +r < 1 . 

b. The H M norm of the closed-loop transfer function is strictly less than a given positive 
scalar / , 

c. The closed-loop value of the cost function (7) exists an upper bound satisfying J <J* , 
then for all admissible uncertainties and possible faults, the given indices, quadratic D 
stabilizability index <&(q,r) , H„ norm bound />0 and cost function performance / >0 
are said to be consistent, state-feedback controller u(lc) = Kx(fc) is said to be satisfactory 
fault-tolerant controller. 

Now, the satisfactory fault-tolerant control problem considered in this paper is stated in the 

following. 

Problem: For the system (1) with actuator failure, given the quadratic D stabilizability index 

<&(q,r) , H„ norm bound y>0 and the cost function (7), determine a control law 

u(fc) = Kx(lc) so that the closed-loop system satisfies criteria (a), (b) and (c) simultaneously. 

3. Main results 

Lemma 1: Consider the actuator fault model (3), for any matrix R = R > and scalar s > , 
if R _1 - el > then 

MRM < M (R- 1 - elj M + e -1 M JJM (8) 

Lemma 2: Consider the system (1) subject to faults, given index <t>(q,r) , if there exists gain 
matrix K and symmetric positive matrix P such that the following matrix inequality 

F 1 A c -ql 

l- \T 2 <° ( 9 ) 

(Ac-ql) -r 2 P 
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holds for all admissible uncertainties and possible faults, then the system (1) is quadratically 

D stabilizable. 

Remark 2: It can be easily shown that in the case when q = , r = 1 , the definition of 

quadratic D stabilizability is reduced to quadratic stabilizability where no closed-loop pole 

constraints is considered. Therefore, Lemma 2 shows that if the uncertain system (1) is 

quadratic D stabilizable, then for some state feedback controllers both quadratic 

stabilizability and pole assignment constraints of the faulty closed-loop system are enforced 

simultaneously. 

Theorem 1: Consider the system (1), for the given index <t>(q,r) , if there exists symmetric 

positive matrix X , matrix Y and scalars s { > 0(i = 1 ~ 3) such that the following linear 

matrix inequality 



s ll 


AX + BY - 


qX 





£ 3 BJ 





* 


2 
-r X 




X 


Y T 


Y 7 


* 


* 




-£{L 








* 


* 




* 


-£ 2 I + £ 3 J 





* 


* 




* 


* 


-ff 3 u 



<0 



(10) 



holds, where £ n = -X + s-^al + s 2 bl + s 3 BJB . Then for all admissible uncertainties and 
possible faults M , the faulty closed-loop system (6) with satisfactory fault-tolerant 
controller u(fc) = Kx(A:) =Mq 1 Y X _1 x(A:) is quadratically D stabilizable. 

Remark 3: Theorem 1 shows us LMI with X and Y, which can be tested with convex 
optimization to decide whether it is solvable, and Matlab LMI Control Toolbox can be 
utilized to solve it. If LMI (13) holds, there must exist state-feedback controller assigning the 
closed-loop poles within <£>(q,r) , namely, the constraint (a) is met. In this case the system (1) 
is said to be robust fault-tolerant state feedback assignable for actuator faults case. 
Lemma 3: Consider the system (1) in fault case and the cost function (7) as well as square 
integrable disturbance co(fc) , if there exists gain matrix K and symmetric positive matrix P 
such that the following matrix inequality 



A£PA C P + C C T C C + Q + K T MRMK + A^PD (y 2 \ D r PDJ D r PA c < 



(11) 



holds for all admissible uncertainties and possible faults M , then the faulty closed-loop 
system is asymptotically stable with an H„ norm-bound y , and the cost function (7) has an 
upper bound 



]<xlvx + r 2 p 2 



(12) 



Remark 4: In some literature on the guaranteed cost control with regional pole constraint 
such as [12], the upper bound of cost function / is that ] <V(0) / r =x Px /r . For 
< r < 1 , it is certainly larger than the one in (12) when co(£) = . So the result here provides 
an improved performance bound. 

Remark 5: Note this upper bound in (12) which depends on the initial condition x . To 
remove the dependence on the initial state, suppose x is arbitrary but belongs to the set 
W = {x e 9?" : x = Uv,v v < 1} , where U is a given matrix. The cost bound in (12) then 
leads to 
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/ < xjpx + y 2 p 2 < 4, ax (u r pu) + r 2 p 2 



Theorem 2: Consider the system (1) and the cost function (7), for the given index <b(q,r) 
and H„ norm-bound index y , if there exists symmetric positive matrix X , matrix Y and 
scalars e { > 0(i = 4 ~ 9) such that the following linear matrix inequality 



-X 





(ax + by) 


a 2i 


* 


- r h 


D T 





* 


* 


-x + f 4 ai + e 5 bi + e 6 B jb t 





* 


* 


* 


S 22_ 



<0 



(13) 



holds, where S 21 =[(CX + EY) ,X,X,Y T ,Y r ,Y T ,Y r ,Y T ,Y r j], 2 22 = diag[-I + s 7 EJE T ,-Q 1 ,-e l l, 

-e 5 I + £ s J,-s 6 J~ ,-s 7 J~ ,-£ g J~ ,£ 9 I-R~ ,-£ 9 l] . Then for all admissible uncertainties and 

possible faults M , the faulty closed-loop system (6) with satisfactory fault-tolerant 

controller u(k) = Kx(A:) = M YX~ x(k) is asymptotically stable with an H„ norm-bound y , 

and the corresponding closed-loop cost function (7) is with / < A max (U T X _1 U) + y 2 j3 2 . 

According to Theorem 1 and 2, the consistency of the quadratic D stabilizability constraint, 
H<» performance and cost function indices for fault-tolerant control is deduced as the 
following optimization problem. 

Theorem 3: Given quadratic D stabilizability index <t>(q,r) , suppose the system (1) is robust 
fault-tolerant state feedback assignable for actuator faults case, then LMIs (10), (13) have a 
feasible solution. Thus, the following minimization problem is meaningful. 



mm(y) : (X,\,y,£ t ) S.t. LMIs (10), (13) 



(14) 



Proof: Based on Theorem 1, if the system (1) is robust fault-tolerant state feedback 
assignable for actuator faults case, then inequality 



A?PA r 



P<0 



has a feasible solution P , K . And existing X > , S > , the following inequality holds 

X [ AcPA c - p] + C c r C c + Q + K r MRMK + SI < (15) 

Then existing a scalar y Q , when y > y , it can be obtained that 

A^PjD^ 2 ! -D T P a D) D T P 1 A C <^I 

where P x = XV . Furthermore, it follows that 

A^PjAc - P a + C C T C C + Q + K T MRMK + A^D [y 2 \ - D^d)" D t P 1 A c < 

Using Schur complement and Theorem 2, it is easy to show that the above inequality is 
equivalent to linear matrix inequality (13), namely, P x , K , y is a feasible solution of LMIs 
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(10), (13). So if the system (1) is robust fault-tolerant state feedback assignable for actuator 

faults case, the LMIs (10), (13) have a feasible solution and the minimization problem (14) is 

meaningful. The proof is completed. 

Suppose the above minimization problem has a solution X L , Y L , s iL , y L , and then any 

index y > y L , LMIs (10), (13) have a feasible solution. Thus, the following optimization 

problem is meaningful. 

Theorem 4: Consider the system (1) and the cost function (7), for the given quadratic D 

stabilizability index <t>(q,r) and H M norm-bound index y>y L , if there exists symmetric 

positive matrix X , matrix Y and scalars e i > 0(i = 1 ~ 9) such that the following 

minimization 



min A + y 2 fl 2 



(16) 



S.t. (i) 


10), (13) 


(ii) 


Al U T 
U X 



<0 



has a solution X min ,Y min , 



L ,A min , then for all admissible uncertainties and possible faults 
M / u(fc) = Kx(fc) = M 1 Y min X m 1 in x(fc) is an optimal guaranteed cost satisfactory fault-tolerant 
controller, so that the faulty closed-loop system (6) is quadratically D stabilizable with an H w 
norm-bound y , and the corresponding closed-loop cost function (7) satisfies 

]<A min +y 2 p 2 - 

According to Theorem 1~4, the following satisfactory fault-tolerant controller design 
method is concluded for the actuator faults case. 

Theorem 5: Given consistent quadratic D stabilizability index 0(q,r) , H m norm index 
y > y L and cost function index / > A min + y j3 , suppose that the system (1) is robust fault- 
tolerant state feedback assignable for actuator faults case. If LMIs (10), (13) have a feasible 
solution X , Y , then for all admissible uncertainties and possible faults M , 
u(fc) = Kx(fc) = Mg YX x(k) is satisfactory fault-tolerant controller making the faulty closed- 
loop system (6) satisfying the constraints (a), (b) and (c) simultaneously. 

In a similar manner to the Theorem 5, as for the system (1) with quadratic D stabilizability, 
H<» norm and cost function requirements in normal case, i.e., M = I , we can get the 
satisfactory normal controller without fault tolerance. 

4. Simulative example 

Consider a satellite attitude control uncertain discrete-time system (1) with parameters as 
follows: 



-2 2 



, B 



1 1 
1 



C = [0.2 0.3], D: 



0.1 

0.5 



,Q = 



"1 


0" 


, R = 


L u 


1 





1 
1 



, a = 0.1, b = 0.2 



Suppose the actuator failure parameters M ; = diag{0A, 0.6} , M M =diag{13, 1.1} . Given the 
quadratic D stabilizability index <1>(0. 5,0.5) , we can obtain state-feedback satisfactory fault- 
tolerant controller (SFTC), such that the closed-loop systems will meet given indices 
constraints simultaneously based on Theorem 5. 
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K Q 



0.5935 3.0187 
-6.7827 -5.6741 



In order to compare, we can obtain the state-feedback satisfactory normal controller (SNC) 
without fault-tolerance. 



K q 



0.4632 2.4951 
-5.4682 -4.9128 



Through simulative calculation, the pole-distribution of the closed-loop system by 
satisfactory fault-tolerant controller and normal controller are illustrated in Figure 1, 2 and 3 
for normal case and the actuator faults case respectively. It can be concluded that the poles 
of closed-loop system driven by normal controller lie in the circular disk 0(0.5,0.5) for 
normal case (see Fig. 1). However, in the actuator failure case, the closed-loop system with 
normal controller is unstable; some poles are out of the given circular disk (see Fig. 2). In the 
contrast, the performance by satisfactory fault-tolerant controller still satisfies the given pole 
index (see Fig. 3). Thus the poles of closed-loop systems lie in the given circular disk by the 
proposed method. 



0.5 -- 
0.4 -- 
0.3 -- 
0.2 -- 
0.1 -- 
-- 
-0.1 -- 
-0.2 - - 
-0.3 - - 
-0.4 - - 
-0.5 - - 



+ 



+ 

4- 

-t+- 



0.2 0.4 0.6 0.8 1 

Fig. 1. Pole-distribution under satisfactory normal control without faults 



5. Conclusion 

Taking the guaranteed cost control in practical systems into account, the problem of 
satisfactory fault-tolerant controller design with quadratic D stabilizability and H„ norm- 
bound constraints is concerned by LMI approach for a class of satellite attitude systems 
subject to actuator failures. Attention has been paid to the design of state-feedback controller 
that guarantees, for all admissible value-bounded uncertainties existing in both the state and 
control input matrices as well as possible actuator failures, the closed-loop system to satisfy 
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the pre-specified quadratic D stabilizability index, meanwhile the H„ index and cost 
function are restricted within the chosen upper bounds. So, the resulting closed-loop system 
can provide satisfactory stability, transient property, Hoc performance and quadratic cost 
performance despite of possible actuator faults. The similar design method can be extended 
to sensor failures case. 




-0.2 0.2 0.4 0.6 0.8 1 

Fig. 2. Pole-distribution under satisfactory normal control with faults 




0.2 0.4 0.6 0.8 1 

Fig. 3. Pole-distribution under satisfactory fault-tolerant control with faults 
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1. Introduction 

Nowadays nearly all the control algorithms are implemented digitally and consequently 
discrete-time systems have been receiving ever increasing attention. However, as to the 
development of nonlinear adaptive control methods, which are generally regarded as smart 
ways to deal with system uncertainties, most researches are conducted for continuous-time 
systems, such that it is very difficult or even impossible to directly apply many well 
developed methods in discrete-time systems, due to the fundamental difference between 
differential and difference equations for modeling continuous-time and discrete-time systems, 
respectively. Even some concepts for discrete-time systems have very different meaning from 
those for continuous-time systems, e.g., the "relative degrees" defined for continuous-time 
and discrete-time systems have totally different physical explanations Cabrera & Narendra 
(1999). Therefore, nonlinear adaptive control of discrete-time systems needs to be further 
investigated. 

On the other hand, the early studies on adaptive control were mainly concerning on the 
parametric uncertainties, i.e., unknown system parameters, such that the designed control 
laws have limited robustness properties, where minute disturbances and the presence of 
nonparametric model uncertainties can lead to poor performance and even instability of 
the closed-loop systems Egardt (1979); Tao (2003). Subsequently, robustness in adaptive 
control has been the subject of much research attention for decades. However, due to 
the difficulties associated with discrete-time uncertain nonlinear system model, there are 
only limited researches on robust adaptive control to deal with nonparametric nonlinear 
model uncertainties in discrete-time systems. For example, in Zhang et al. (2001), parameter 
projection method was adopted to guarantee boundedness of parameter estimates in presence 
of small nonparametric uncertainties under certain wild conditions. For another example, 
the sliding mode method has been incorporated into discrete-time adaptive control Chen 
(2006). However, in contrast to continuous- time systems for which a sliding mode controller 
can be constructed to eliminate the effects of the general uncertain model nonlinearity, for 
discrete-time systems, the uncertain nonlinearity is normally required to be of small growth 
rate or globally bounded, but sliding mode control is yet not able to completely compensate 
for the effects of nonlinear uncertainties in discrete-time. As a matter of fact, unlike in 
continuous-time systems, it is much more difficulty in discrete-time systems to deal with 
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nonlinear uncertainties. When the size of the uncertain nonlinearity is larger than a certain 
level, even a simple first-order discrete-time system cannot be globally stabilized Xie & Guo 
(2000). In an early work on discrete-time adaptive systems, Lee (1996) it is also pointed 
out that when there is large parameter time-variation, it may be impossible to construct 
a global stable control even for a first order system. Moreover, for discrete-time systems, 
most existing robust approaches only guarantee the closed-loop stability in the presence of 
the nonparametric model uncertainties, but are not able to improve control performance by 
complete compensation for the effect of uncertainties. 

Towards the goal of complete compensation for the effect of nonlinear model uncertainties 
in discrete-time adaptive control, the methods using output information in previous steps 
to compensate for uncertainty at current step have been investigated in Ma et al. (2007) 
for first order system, and in Ge et al. (2009) for high order strict-feedback systems. We 
will carry forward to study adaptive control with nonparametric uncertainty compensation 
for NARMA system (nonlinear auto-regressive moving average), which comprises a general 
nonlinear discrete-time model structure and is one of the most frequently employed form in 
discrete-time modeling process. 

2. Problem formulation 

In this chapter, NARMA system to be studied is described by the following equation 

n m 

y(k + n) = '£8f<p i (y(k + n - i)) + £ gju(k - m + j) + v(z(k - t)) (1) 

i=l /=1 

where y(k) and u(k) are output and input, respectively. Here 

y_(k) = [y(k),y(k-l) y(k-n + l)] T (2) 

u(k) = [u(k - l),u(k - 2), . . . ,u(k - m + 1)] T (3) 

and z(k) = [y T (k),u T {k - 1)} T . And for / = 1,2, ■■■ ,n, </>,(■) : R" -)■ RP' are known 

vector-valued functions, dj = [9^1, ■ ■ ■ , Oi,p t ], and g: are unknown parameters. And the last 
term v(z(k — t)) represents the nonlinear model uncertainties (which can be regarded as 
unmodeled dynamics uncertainties) with unknown time delay t satisfying < T m i n < t < 
Tmax for known constants T m i n and T max . The control objective to make sure the boundedness 
of all the closed-loop signals while to make the output y(k) asymptotically track a given 
bounded reference y* (k) . 

Time delay is an active topic of research because it is frequently encountered in engineering 
systems to be controlled Kolmanovskii & Myshkis (1992). Of great concern is the effect 
of time delay on stability and asymptotic performance. For continuous-time systems 
with time delays, some of the useful tools in robust stability analysis have been well 
developed based on the Lyapunov's second method, the Lyapunov-Krasovskii theorem and 
the Lyapunov-Razumikhin theorem. Following its success in stability analysis, the utility 
of Lyapunov-Krasovskii functionals were subsequently explored in adaptive control designs 
for continuous-time time delayed systems Ge et al. (2003; 2004); Ge & Tee (2007); Wu (2000); 
Xia et al. (2009); Zhang & Ge (2007). However, in the discrete-time case there dos not exist 
a counterpart of Lyapunov-Krasovskii functional. To resolve the difficulties associated with 
unknown time delayed states and the nonparametric nonlinear uncertainties, an augmented 
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states vector is introduced in this work such that the effect of time delays can be canceled at 
the same time when the effects of nonlinear uncertainties are compensated. 
In the NARMA system described in (1), we can see that there is a "relative degree" n which 
can be regarded as response delay from input to output. Thus, the control input at the fcth 
step, u(k), will actually only determine the output at n-step ahead. The n-step ahead output 
y(k + n) also depends on the following future outputs: 

y(k + l),y(k + 2),...,y(* + n-2),y(* + n-l) (4) 

and ideally the controller should also incorporate the information of these states. However, 
dependence on these future states will make the controller non-causal! 

If system (1) is linear, e.g., there is no nonlinear functions (pi, we could find a so called 
Diophantine function by using which system (1) can be transformed into an M-step predictor 
where y(k + n) only depends on outputs at or before the fc-th step. Then, linear adaptive 
control can be designed under certainty equivalence principal to emulate a deadbeat controller, 
which forces the n-step ahead future output to acquire a desired reference value. However, 
transformation of the nonlinear system (1) into an M-step predictor form would make the 
known nonlinear functions and unknown parameters entangled together and thus not 
identifiable. Thus, we propose future outputs prediction, based on which adaptive control can 
be designed properly. 
Throughout this chapter, the following notations are used. 

• HI! denotes the Euclidean norm of vectors and induced norm of matrices. 

• Z+ represents the set of all integers which are not less than a given integer t. 

• 0r q i stands for (j-dimension zero vector. 

• A := B means that A is defined as B. 

• (") and (") denote the estimates of unknown parameters and estimate errors, respectively. 

3. Assumptions and preliminaries 

Some reasonable assumptions are made in this section on the system (1) to be studied. In 
addition, some useful lemmas are introduced in this section to facilitate the later control 
design. 

Assumption 3.1. In system (1), the functional uncertainty !/(•), satisfies Lipschitz condition, i.e., 
||v(ei) — v(£2)|| < Lvll^i — £i\\i Vei,62 £ R n , where L v < A* with A* being a small number 
defined in (58). The system functions (pj{-), i = 1,2, ... , n, are also Lipschitz functions with Lipschitz 
coefficients L,. 

Remark 3.1. Any continuously derivable function is Lipschitz on a compact set, refer to Hirsch & 
Smale (1974) and any function with bounded derivative is globally Lipschitz. As our objective is 
to achieve global asymptotic stability, it is not stringent to assume that the nonlinearity is globally 
Lipschitz. 

In fact, Lipschitz condition is a common assumption for nonlinearity in the control community 
Arcak et al. (2001); Nesic & Laila (July, 2002); Nesic & Teel (2006); Sokolov (2003). In addition, it 
is usual in discrete-time control to assume that the uncertain nonlinearity is of small Lipschitz 
coefficient Chen et al. (2001); Myszkorowski (1994); Zhang et al. (2001); Zhu & Guo (2004). 
When the Lipschitz coefficient is large, discrete-time uncertain systems are not stabilizable as 
indicated in Ma (2008); Xie & Guo (2000); Zhang & Guo (2002). Actually, if the discrete-time 
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models are derived from continuous-time models, the growth rate of nonlinear uncertainty 
can always be made sufficient small by choosing sufficient small sampling time. 

Assumption 3.2. In system (1), the control gain coefficient g m of current instant control input u(k) 
is bounded away from zero, i.e., there is a known constant g > such that \g m | > g , and its sign is 
known a priori. Thus, without loss of generality, we assume g m > 0. 

Remark 3.2. It is called unknown control direction problem when the sign of the control gain is 
unknown. The unknown control direction problem of nonlinear discrete-time system has been well 
addressed in Ge et al. (2008); Yang et al. (2009) but it is out the scope of this chapter. 

Definition 3.1. Chen & Narendra (2001) Let Xi(fc) and X2{k) be two discrete-time scalar or vector 
signals, Vfc G Z ( + ,for any t. 

• We denote x\ (k) = 0[x2(k)\, if there exist positive constants m\, mi and fcg such that \\x\ (k) || < 
ffi! max t( < t ||x 2 (fc')ll + mi, VA: > k . 

• We denote x\(k) = o[x2{k)], if there exists a discrete-time function a(k) satisfying Hni^coU.^) = 
and a constant kg such that \\x\(k)\\ < a(k) max^ot ||x2(fc')||, Vfc > kg. 

• We denote x\(k) ~ X2CO if they satisfy x\(k) = 0[x2(k)] and X2(k) = 0[x\(k)]. 
Assumption 3.3. The input and output of system (1) satisfy 

u(k) = 0[y(k + n)} (5) 

Assumption 3.3 implies that the system (1) is bounded-output-bounded-input (BOBI) system 

(or equivalently minimum phase for linear systems). 

For convenience, in the followings we use 0[1] and o[l] to denote bounded sequences and 

sequences converging to zero, respectively. In addition, if sequence y(k) satisfies y(k) = 

0[x(k)] or y(k) = o[x(k)], then we may directly use 0[x(k)] or o[x(k)] to denote sequence 

y(k) for convenience. 

According to Definition 3.1, we have the following proposition. 

Proposition 3.1. According to the definition on signal orders in Definition 3.1, we have following 
properties: 

(i) 0[xi(k + T)]+0[xi(k)] ~ 0[x x (k + t)],Vt>0. 

(ii) Xl (k + T)+o [xj (k)] ~ x 1 (k + t), Vt > 0. 

(Hi) o[ Xl (k + t)} + o[ Xl (k)} ~o[x 1 (/c + t)] / Vt>0. 

(iv) o[ Xl (k)}+o[x 2 (k)} ~ o[|x!(fc)| + \x 2 (k)\]. 

(v) o[0[x 1 (k)]]~o[x 1 (fc)] + 0[l]. 

(vi) lfx\{k) ~ x 2 (k) and\m\ k ^ oa \\x 2 (k)\\ = 0, then lirnj^oo ||xx(fc)|j = 0. 

(vii) Ifxi(k) = o[xi(k)] +o[i\,thenlxaxj c ^ oa \\xi(k)\\ =0. 

(via) Letx 2 {k) = x x (fc) +o[x r (k)]. Ifx 2 (k) = o[l],then]sm k -^ oa \\x\{k)\\ =0. 

Proof. See Appendix A. ■ 

Lemma 3.1. Goodwin et al. (1980) (Key Technical Lemma) For some given real scalar sequences s(k), 
b\(k), b2{k) and vector sequence cr(k), if the following conditions hold: 
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(i) lim^oo f , 1 (j t ) +i , 2 ( t ) (7 .T(/ t )o.( t ) = 0' 

(H) bi(k) = 0[1] and b 2 {k) = 0[1], 

(m) tr(jfc) =0[s(fc)]. 

T/ie«, we fraue 

fl) limj : _5. o s(k) = 0, and b) a(k) is bounded. 

Lemma 3.2. Define 

Z(k) = [z(fc-T max ),...,z(fc-T),...,z(fc-T min )] (6) 



and 



such that 



and 



J fc = argnun ||Z(k)-Z(!)|| (7) 

Kk—n 



Z (k) = [z(/fc-Tmax)/...,z(k-T) / ... / z(Zjt-Tinj n )] (8) 



AZ(fc) = Z(fc) - Z(/ fc ) (9) 



Tfen, if ||Z(fc)|| is bounded we have ||AZ(fc)|| — > as well as ||v(z(fc — t)) — v(z(l k — t))|| — > 0. 
Proof. Given the definition of l k in (7), it has been proved in Ma (2006); Xie & Guo (2000) that 
the boundedness of sequence Z(fc) leads to ||AZ(fc)|| — > 0. As < ||v(z(fc — t)) — v(z(l k — 
t))|| < ||AZ(fc) ||, it is obvious that ||v(z(fc-r)) -v(z(l k -r))\\ -> as fc -> oo. a 

According to the definition of AZ(/c) in (9) and Assumption 3.1, we see that 

\v(z(k - t)) - i/(z(/ Jt - t))| < MAZ(/c))|| (10) 

The inequality above serves as a key to compensate for the nonparametric uncertainty which 
will be demonstrated later. 

4. Future output prediction 

In this section, an approach to predict the future outputs in (4) is developed to facilitate control 
design in next section. To start with, let us define an auxiliary output as 

y a (k + n-l) = f^eJct> 1 (y(k + n-i))+v(z(k-T)) (11) 

i=l 

such that (1) can be rewritten as 

m 

y{k + n) =y a {k + n-l) + Y J gju(k -m + j) (12) 

7=1 

It is easy to show that 

y„(k + n-l) = y a (k + n - 1) -y a (l k + n-l) +y a (l k + n - 1) 

a m 

= E 0f[fc(y(* + « - 0) - 4>iWk + n- /))] - E 8j u ( l k -m+j) 
i=l /=1 

+y(l k + n) + (v(z(k - t)) - v(z(l k - t)) (13) 
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For convenience, we introduce the following notations 

Acf>i{k + n- i) = 4>i(y(k + n-i))- 4>,{y{l k + n- j)) (14) 

Au(k — m + j) = u(k — m + j) — u(l k — m + j) 

Av(k - t) = v(z(k - t)) - v(z(l k - t)) (15) 

for i = 1, 2, ... ,n and j = 1,2, . . . ,m. 
Combining (12) and (13), we obtain 

n m 

y{k + n) = J^efAcp i (k + n-i) + J^gjAu(k- m + j) +ij{l k + n) + Av(k- r) (16) 
i=\ y=i 

Step 1: 

Denote §i(k) and g;(k) as the estimates of unknown parameters 0, and g; at the kth step, 
respectively. Then, according to (16), one-step ahead future output y(k + 1) can be predicted 
at the /cth step as 

n m 

y(k + l|fc) = £ §J(k -n + 2)A&(fc - i + 1) + £ f,(fc - « + 2)Au(fc -m + j-n + 1) 
i=\ j=\ 

+y{h-n+l + n) (17) 

Now, based on y(k + l\k), we define 

Afo(fc + l|fc) = fc(£(* + l|fc)) - <h(yGjt-„+ 2 + n - 1)) (18) 

which will be used in next step for prediction of two-step ahead output and where 

y(fc + l|fc) = [y(fc + l|fc),y(fc) y(k-n + 2)] T (19) 

Step 2: By using the estimates #,(k) and gj(k) and according to (16), the two-step ahead future 
output y(k + 2) can be predicted at the A:th step as 

y(k + 2\k) = §J{k -n + 3)A&(fc + l|fc) + £ 0, r (*r - n + 3)A<fc(* - i + 2) 

1=2 

/!/ 

+ E 8j(k - n + 3)Aw(fc - m + / - n + 2) + y(l k - n+2 + n) (20) 

7=1 

Then, by using y(fc + l|fc) and y(fc + 2\k), we define 

Afo(* + 2\k) = fr(£(* + 2\k)) - <pi(y(k-n+3 + n - 1)) 

A<fe(fc + l|fc) =4> 2 (y(k+l\k))-cp 2 (y(l k _ ll+3 + n-2)) (21) 
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which will be used for prediction in next step and where 



£(* + 2\k) = [y(k + 2\k),y(k + l\k),y(k), . . . ,y(k - n + 3)] T (22) 

Continuing the procedure above, we have three-step ahead future output prediction and so 
on so forth until the (n — l)-step ahead future output prediction as follows: 
Step (n — 1): The (n — l)-step ahead future output is predicted as 

y(k + n - l\k) = £ S[(k)Af t (k + n - 1 - i\k) + £ §J(k)A<Pi(k - (i - (n - 1))) 

i=\ i=n-\ 

m 

+ E l/WM* -m + j-l)+ y(Z t _j + ft) (23) 

7=1 

where 

A$i{k + l\k) = (t>i{l{k + l\k))-<Pi(y(l k _ n+i+ i + n-i)) (24) 

for/' = 1,2,..., n -2 and/ = 1,2, ...,«- r - 1. 

The prediction law of future outputs is summarized as follows: 

!-l R 

y(k + l\k) = £ 0?( fc - « + ' + W,-(* + / - i\k) + E ^ T (fc - n + 1 + l)A^(Jt - (i - Z)) 

[=1 ! = / 

+ E ^;( fc ) A "( fc - m - n + /+;) + y(/jt- n +l + ») (25) 

i=i 

for/ = l,2,...,n-l. 

Remark 4.1. Note f/wf 0,-(fc — n + / + 1) and ^y(fc — ft + / + 1) instead o/6,(fc) and g; W <we wsed 
m f/ze prediction law of the l-step ahead future output. In this way, the parameter estimates appearing 
in the prediction of y(k + l\k) and y(k + l\k + 1) are at the same time step, such that the analysis of 
prediction error will be much simplified. 

Remark 4.2. Similar to the prediction procedure proposed in Yang et al. (2009), the future output 
prediction is defined in such a way that the j-step prediction is based on the previous step predictions. 
The prediction method Yang et al. (2009) is further developed here for the compensation of the effect 
of the nonlinear uncertainties v(z(k — t)). With the help of the introduction of previous instant 1% 
defined in (7), it can been seen that in the transformed system (16) that the output information at 
previous instants is used to compensate for the effect of nonparametric uncertainties v(z(k — t)) at the 
current instant according to (15). 

The parameter estimates in output prediction are obtained from the following update laws 

a p (fc)7pA<fc(/c-i + l)#(fc+l|fc) 



0i(k + l) =9i(k-n + 2)- 
gj(k + l)=gj(k-n + 2)- 



D p (k) 
a p (k)y p Au(k -m + j-n+ l)y(k + l|/c) 



D p (fc) 
1,2,..., n, j = 1, 2, . . . , m (26) 
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with 

y(k + l\k) = y(k + l\k)-y(k + l) 

n ni 

D p (k) = 1 + £ || A<pi(k -i + l)\\ 2 + E A " 2 ( fc - « + /- n + 1) (27) 

C 1 _ A||_Z(fc-H+l)|| 
\y(k+\\k)\ ' 
«pW = < if |y(fc + l|fc)| > A||AZ(fc-« + l)|| ( 28 ) 

[ otherwise 

0,(0) = W , gj(0) = (29) 

where < y„ < 2 and A can be chosen as a constant satisfying L v < A < A*, with A* defined 
later in (58). 

Remark 4.3. The dead zone indicator a p (k) is employed in the future output prediction above, which is 
motivated by the work in Chen et al. (2001). In the parameter update law (38), the dead zone implies that 
in the region \y(k + l|fc)| < A||AZ(fc — n + 1)||, the values of parameter estimates at the (k + l)-th 
step are same as those at the (k + n — 2)-th step. While the estimate values will be updated outside of 
this region. The threshold of the dead zone will converge to zero because lims._5.c0 ||AZ(fe — n + 1)|| = 
0, which will be guaranteed by the adaptive control law designed in the next section. The similar dead 
zone method will also be used in the parameter update laws of the adaptive controller in the next section. 

With the future outputs predicted above, we can establish the following lemma for the 
prediction errors. 

Lemma 4.1. Define y(k + l\k) = y(k + l\k) — y(k + I) , then there exist constant c\ such that 

\y(k + l\k)\ = o[0[y(k + l)]] + AA s (k,l), l = \,2,...,n-\ (30) 

where 

As(M) = max {||AZ(fc-n + fc')ll) (31) 

1 <k'<r 

Proof. See Appendix B. ■ 

5. Adaptive control design 

By introducing the following notations 

e = [e 1 ,e 2 ,...,9 n ] 

4>(k + n-l) = [k<p l (yJ,k + n-l))A<p 2 (y_(k + n-2)),...,k<p n (yjk))} T 

g= [gl,g2,---,gm] T 

u(k) = [Au 1 {k-m + l),Au 2 {k-m + 2),...,Au m (k)] T (32) 

we could rewrite (16) in a compact form as follows: 

y(k + n) = 9 T 4>(k + n - 1) + g T il(k) + y(l k + n) + Av(k - t) (33) 
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Define 6(k) and g(k) as estimate of 8 and g at the kth step, respectively, and then, the controller 
will be designed such that 

y*{k + n) = § T (k)${k + n-l)+g{k) T u{k)+y{l k + n) (34) 

Define the output tracking error as 

e(*)=y(k)-y*(fc) (35) 

A proper parameter estimate law will be constructed using the following dead zone indicator 
which stops the update process when the tracking error is smaller than a specific value 

ac{k) = { 1 ~ *ll^y ^ if \e(k)\ > Al|AZ(fc-n)|| + lj8 (fc- 1)| ^ 

y otherwise 

where 

jB(fc-l)=3 T (fc-n)(#(fc-l)-#(fc-l)) (37) 

and A is same as that used in (28). 

The parameter estimates in control law (34) are calculated by the following update laws: 

kk ) = e { k-n ) + ^^- 1 h (k ) 

m=§(k-n) + ^ k W- n K (k) (38) 



with 



D c (k) = 1 + \\$(k - 1) || 2 + \\u(k -n)\\ 2 (39) 



and < 7 C < 2. 

Remark 5.1. To explicitly calculate the control input from (34), one can see that the estimate of g m , 
gm{k), which appears in the denominator, may lead to the so called "controller singularity" problem 
when the estimate g m (k) falls into a small neighborhood of zero. To avoid the singularity problem, we 
may take advantage of the a priori information of the lower bound of g m , i.e. g , to revise the update 
law of gm{k) in (38) as follows: 

*i, v \ ±d \ , 7ca c {k)u(k-n) 
g '(k) = g(k -n) + ^— ^ '-e(k) 

\gr(k) otherwise K ' 

(41) 

where 

g'(k) = [g[(k),g' 2 (k) g' m {k)] 

gr(k) = ig[(k),g' 2 (k) gJ T (42) 

In (40), one can see that in case where the estimate of control gain g m (k) falls below the known 
lower bound, the update laws force it to be at least as large as the lower bound such that the potential 
singularity problem will be solved. 
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6. Main results and closed-loop system analysis 

The performance of the adaptive controller designed above is summarized in the following 
theorem: 

Theorem 6.1. Under adaptive control law (34) with parameter estimation law (38) and with 
employment of predicted future outputs obtained in Section 4, all the closed-loop signals are guaranteed 
to be bounded and, in addition, the asymptotic output tracking can be achieved: 

lim|y(fc)-y*(fc)|=0 (43) 

K— 7-00 

To prove the above theorem, we proceed from the expression of output tracking error. 
Substitute control law (34) into the transformed system (33) and consider the definition of 
output tracking error in (35), then we have 

e(k) = -§ T (k - n)${k - 1) - g T {k - n)u(k - n) - fi{k - 1) + Av(k -n-r) (44) 

where 6(k) = 6(k) — 9 and g(k) = g(k) — gAv(k — n — r) satisfies 

||Av(fc-«-T)|| < A||AZ(fc-n)|[ (45) 

From the definition of dead zone indicator a c (k) in (36), we have 

a c (k)i\e(k)\(A\\AZ(k-n)\\ + \(3(k-l)\)-e 2 (k)] = -a 2 c (k)e 2 (k) (46) 

Let us choose a positive definite Lyapunov function candidate as 

v c (k)= E (\\Hi)f + M(i)f) (47) 

l=k-n+l 
and then by using (46) the first difference of the above Lyapunov function can be written as 

AV c (k) = V c (k)-V c (k-l) 

< 8 T {k)6(k) - § T {k - n)6(k - n) + g 2 {k) - g 2 {k - n) 

A(k)^ 2 e 2 (k) 



||^(fc-l)f + ||fl(fc-n)||' 1 



D 2 (k) 

2a c (k)j c 



< 



+ [6 1 (k-n)4>(k-l)+g 1 (k-n)u(k-n)}e(k) . 

a 2 (kh 2 e 2 (k) 2a c (k) 7c e 2 (k) 
D c (k) D c (k) 

2a c (k) 7c \e(k)\(A\\AZ(k-n)\\ + \[3(k^l)\) 

D c (k) 
7c (2- 7c )a 2 (k)e 2 (k) 
D c (k) 



(48) 



Noting that < y c < 2, we have the boundedness of V c (k) and consequently the boundedness 
of 9(k) and g(k). Taking summation on both hand sides of (48), we obtain 

E7c(2-7c)*3^<Vc(O)-V c («0 

k=0 Uci - K > 
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which implies 

, fl?(fc)e 2 (fc) 

lim n m = ° 49 

/c^oo D c (k) 

Now, we will show that equation (49) results in lirrtj-^oo a c (k)e(k) = using Lemma 3.1, the 
main stability analysis tool in adaptive discrete-time control. In fact, from the definition of 
dead zone a c (k) in (36), when \e(k)\ > A||AZ(fc — n)\\ + |/S(fc— l)|,wehave 

a c (k)\e(k)\ = |e(fc)|-A||AZ(Jt-n)||-||S(fc-l)| >0 
and when \e(k)\ < A|| Az(/c — n)\\ + \fi{k — l)|,wehave 

a c {k)\e{k)\ =0> |e(fc)|-A||AZ(fc-n)||-|/3(fc-l) 

Thus, we always have 

|e(*)|-A||AZ(k-n)||-|j3(*-l)| < a c (k)\e(k)\ (50) 

Considering the definition of fl(k — 1) in (37) and the boundedness of 9(k), we obtain that 

p(k-l)=o[0[y(k)}}. 

Since y(k) ~ e(k), we have fi(k — 1) = o[0[e(fc)]]. According to the Proposition 3.1, we have 

|y(*)| <C 1 max{\e{k')\} + C 2 

k'<k 

< Q max{| e (fc')| - A||AZ(fc' - n)\\ - \{S{k> - 1)| 

k'<k 
+A||AZ(fc'-n)|| + |/3(fc'-l)|} + C 2 

< Q max{« c (fc')|e(fc') I } + AQ max{ || AZ(fc' - n) || } + Q max{ |£(fc' - 1) |} 

Jc'<Jc A:'<fc /c'<Jc 

+C 2 , Vfc e Z+„ (51) 

According to Lemma 4.1 and Assumption 3.1, there exits a constant Ca such that 

|/3(fc + n-l)| <o[0[y(fc + n - 1)]] + A C/ jA s (fc,n - 1) (52) 

Considering AZ(fc) defined in (9) and A s (k,m) defined in (31), Lemma (3.3), and noting the 
fact l^ < k — n, there exist constants c 2 i, c Zf i, c S/ \ and c S/ 2 such that 

AZ(Jfc-n) < c Z/1 max{|y(fc')|} + c Z/2 (53) 

k'<k 

A s (fc,n-1) = max {||Z(fc-n + fc') - Z(l k _ n+k ,)\\} 
l<k'<n—l 

< c S/1 max{|y(fc' + «-l)|} + c S/2 (54) 

k'<k 

According to the definition of o[] in Definition 3.1, and (52), (54), it is clear that Vfc 6 Z_„ 

|jS(fc + n-l)| < o[0[y(fc + n-l)]]+Ac^A s (/c,n-l) 

< (a(fc)cgi +Ac«jc S/1 )max{|y(fc' + n-l)|} + a(fc)c« 2 + AcflC S/ 2 (55) 
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where lim^co u(k) = 0, and Cri and Ca 2 ar e positive constants. Since limj-^oo ot(k) = 0, for 
any given arbitrary small positive constant e\, there exists a constants k\ such that u(k) < e\, 
Vfc > k\ . Thus, it is clear that 

\fi(k + n - 1) < {e\Cai + Ac«c sl )max{|i/(fc' +w-l)|} + £\Ca 2 + hcac S/2 , V/c > k x (56) 

k'<k 

From inequalities (51), (53), and (56), it is clear that there exist an arbitrary small positive 
constant e 2 and constants C3 and C4 such that 

max{|y(fc')|} < Q max{fl c (fc')|e(fc')l} + (AC 3 + e 2 ) max{|y(fc')|} + C 4 , fc > k x (57) 

fc'<fc Jr'<J: k'<k 



which implies the existence of a small positive constant 

— < 

such that 



A* = — - - (58) 



max^),} < I _^_^ {Bc (^) k (»/)| }+T _^_ / fc>fcl (59) 

holds for all A < A*, where C3 = (c c c 2/1 + cac s r \)C\, e 2 = £\CaiC\ and C4 = C 2 + eiCa j2 C\ + 
Ac c c Z/2 C\ + Xcac Si2 Ci. Note that inequality (59) implies y (k) = 0[a c (k)e(k)]. From <p{y{k + 
n — 1)) defined in (32) and Assumption 3.1, it can be seen that (p(y(k — 1)) = 0[y(k — 1)]. 
According to the definition of D c (k) in (39), y(k) ~ e{k), ^_„ < k — In, the boundedness of 
y* (fc) , and (53), we have 

DUk) <l + ||^(fc_l)|| + | fl (fc_n)| 
= 0[y(k)\ = 0{a c {k)e{k)\ 

Then, applying Lemma 3.1 to (49) yields 

lim a c {k)e(k) = (60) 

k^co 

From (59) and (60), we can see that the boundedness of y(k) is guaranteed. It follows that 
tracking error e(k) is bounded, and the boundedness of u(k) and z(k) in (75) can be obtained 
from (5) in Lemma 3.3, and thus all the signals in the closed-loop system are bounded. Due to 
the boundedness of z(k), by Lemma 3.2, we have 

lim ||AZ(fc)|| = (61) 

k->co 



which further leads to 



lim ||A s (fc,n-l)|| =0 (62) 

k^co 



Next, we will show that lim^^ a c (k)e(k) = implies limj.^,^ e(k) = 0. In fact, considering 
(52) and noting that y(k) ~ e(k) ~ e(k), it follows that 

||3(fc-l)| <o[0[e(k)]] + \c p A s (k-n,n-l) (63) 
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which yields 

\e(k)\ - |/3(fc - 1)| + A C/5 A s (fc - n,n - 1) > \e(k)\ - o[0[e(k)}} 

> (1 - «(*)»!!) |e(Jfc) | - a(fc)m 2 (64) 

according to Definition 3.1, where mj and ni2 are positive constants, and limj-^ co a(fc) = 0. 
Since liim-^co a(fc) = 0, there exists a constant A; 2 such that a(k) < l/m\, V/c > A; 2 . Therefore, 
it can be seen from (64) that 

|e(*)|-|/S(fc-l)| + AcpA s (*-n,n-l) + a(fc)m 2 > (1 - cc{k)m{)\e{k)\ > 0, \/k > k 2 (65) 

From (50), it is clear that 

\e{k) | - |j8(* - 1)| + A C/3 A s (fc - n, k - 1) + a(k)m 2 
< a c {k)\e{k) \ + A||AZ(fc - n)\\ + AcpA s {k - n,n - 1) + a(fc)m 2 (66) 

which implies that limj.^co e (^) = according to (60)-(62), and (65), which further yields 
limj-^oo e(k) = because of e(k) ~ e(k). This completes the proof. ■ 

Remark 6.1. The underlying reason that the asymptotic tracking performance is achieved lies in that 
the uncertain nonlinear term v(k — n — t) in the closed-loop tracking error dynamics (44) will converge 
to zero because limjt_ ) . 00 ||AZ(fc)|| = as shown in (61). 

7. Further discussion on output-feedback systems 

In this section, we will make some discussions on the application of control design technique 
developed before to nonlinear system in lower triangular form. The research interest of 
lower triangular form systems lies in the fact that a large class of nonlinear systems can 
be transformed into strict-feedback form or output-feedback form, where the unknown 
parameters appear linearly in the system equations, via a global parameter-independent 
diffeomorphism. In a seminal work Kanellakopoulos et al. (1991), it is proved 
that a class of continuous nonlinear systems can be transformed to lower triangular 
parameter-strict-feedback form via parameter-independent diffeomorphisms. A similar result 
is obtained for a class of discrete-time systems Yeh & Kokotovic (1995), in which the geometric 
conditions for the systems transformable to the form are given and then the discrete-time 
backstepping design is proposed. More general strict-feedback system with unknown control 
gains was first studied for continuous-time systems Ye & Jiang (1998), in which it is indicated 
that a class of nonlinear triangular systems T\g proposed in Seto et al. (1994) is transformable 
to this form. The discrete-time counterpart system was then studied in Ge et al. (2008), in 
which discrete Nussbaum gain was exploited to solve the unknown control direction problem. 
In addition to strict-feedback form systems, output-feedback systems as another kind of 
lower-triangular form systems have also received much research attention. The discrete-time 
output-feedback form systems have been studied in Zhao & Kanellakopoulos (2002), in 
which a set of parameter estimation algorithm using orthogonal projection is proposed and 
it guarantees the convergence of estimated parameters to their true values in finite steps. In 
Yang et al. (2009), adaptive control solving the unknown control direction problem has been 
developed for the discrete-time output-feedback form systems. 

As mentioned in Section 1, NARMA model is one of the most popular representations of 
nonlinear discrete-time systemsLeontaritis & Billings (1985). In the following, we are going to 
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show that the discrete-time output-feedback forms systems are transformable to the NARMA 
systems in the form of (1) so that the control design in this chapter is also applicable to the 
systems in the output-feedback form as below: 

C x,(k + l) = 6j<p i (x 1 (k))+g,x i+1 (k) + v,(x 1 (k)), i = l,2,...,n-l 

I x n (k + l) =el4> n (x 1 (k))+g n u(k) + v n (x 1 (k)) (67) 

I y(*) = *i(*) 

where Xj(k) £ R, i = 1, 2, ...,n are the system states, n > 1 is system order; u(k) G -R, 
y(k) 6 R is the system input and output, respectively; #, are the vectors of unknown constant 
parameters; g, G R are unknown control gains and g, ^= 0; cpi(-), are known nonlinear vector 
functions; and u,() are nonlinear uncertainties. 

It is noted that the nonlinearities (pi(-) as well as Vu.-. depend only on the output y(k) = *i(fc), 
which is the only measured state. This justifies the name of "output-feedback" form. 
According to Ge et al. (2009), for system (67) there exist prediction functions F H _,(-) such that 
y{k + n — i) = F n _j(y(k),u(k — /')), i = 1,2, . . . ,n — 1, where 

y(k) = [y(k),y(k - l),...,y{k - n + 1)] T (68) 

u(k - i) = [u{k - i),u(k -i-l),...,u(k-n + l)] r (69) 

By moving the z'th equation in — i) step ahead, we can rewrite system (67) as follows 

' xi{k + n)= 6j(pi(y(k + n - 1)) + g\x 2 {k + n - 1) + v\{y{k + n - 1)) 
x 2 (k + n - 1) = 9jcj)2{y(k + n - 2)) + g 2 x 3 {k + n-2) + v 2 {y{k + n - 2)) 

x„(/c + l) = e%(p„(y(k)) +gnu(k) + v„{y{k)) 
Then, we submit the second equation to the first and obtain 

Xl (k + n)= e1<pi (y(k + n - 1)) + gl ejcp 2 (y(k + n - 2)) 

+£ig2* 3 (fc + n - 2) + t'i(y(fc + n - 1)) +^iU2(F„-2(y(fc),«(fc - 2)) (71) 

Continuing the iterative substitution, we could finally obtain 

y(k + n) = f; 0$fc(y(* + n - f)) + gu(k) + v(z(k)) (72) 



(70) 



where 



and 



i-1 

0/1=01,0/5=0*11*/' ; = 2 ' 3 '---' M 
7=1 

^=1- */,=n^ ' = 2 '3 «, g = fl« ( 7 3) 

7=1 7=1 



v(zW) = E*/,^( Z W)' z « = [y T W,M T (fc-i)] T (74) 

1=1 
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with 

v,(z(k)) = Vl (y(k + n - i)) = Vi(F n _i{y{k),u(k - i))), i = 1,2, . . . ,n -1, 
v n (z(k)) = v n {y(k)) (75) 

with z(/c) defined in the same manner as in (1). Now, it is obvious that the transformed 
output-feedback form system (72) is a special case of the general NARMA model (1). 

8. Study on periodic varying parameters 

In this section we shall study the case where the parameters 9j and gj, i = 1,2, . . . ,n, j = 
1, 2, . . . , m in (1) are periodically time-varying. The lih element of 0; (k) is periodic with known 
period Njj and the period of gi(k) is N g j, i.e. 0{j(k) = 0,y(fc — Ny) and gj(k) = gj(k — N g j) for 
known positive constants Ny and 2SL;, / = 1,2, ... , p,-. 

To deal with periodic varying parameters, periodic adaptive control (PAC) has been developed 
in literature, which updates parameters every N steps, where N is a common period such 
that every period IV, j and N„; can divide N with an integer quotient, respectively. However, 
the use of the common period will make the periodic adaptation inefficient. If possible, the 
periodic adaptation should be conducted according to individual periods. Therefore, we will 
employ the lifting approach proposed in Xu & Huang (2009). 

Firstly, we define the augmented parametric vector and corresponding vector-valued 
nonlinearity function. As there are Ny different values of the /th element of 6, at different 
steps, denote an augmented vector combining them together by 

®i,l = [0y,l/ 6i,j,2> ■ ■■' 8i,j,N u ] (76) 

with constant elements. We can construct an augmented vector including all p, periodic 
parameters 

©/ = [6i t i/ 0i,2> ■ ■ ■ i %p ( ] = [01,1,1/ ■ ■ ■ / ^,i,n u - ■ ■ ■ - 0(, P/ ,i/ ■ ■ -'OhpiMjJ ( 77 ) 

with all elements being constant. Accordingly, we can define an augmented vector 

^{yjk + n- 1)) = fe(y(fc + n - 1)),. . .,ft, p ,(y(fc + n - 1))} T (78) 

where ^,(y(fc+ n - 1)) = [0, . . . ,0,<ft(y(fc + n - i)),Q, . . .,0} T e -R N '-' and the element <p { (k) 
appears in the ijth position of cf»i f i{y{k + n — 1)) only when k = sNjj + q, for i = 1,2, . . .,Ny. It 
can be seen that n functions <pi{k), rotate according to their own periodicity, N, j, respectively. 
As a result, for each time instance k, we have 

ej(k)cp l ( l (k + n-i))=el® l (y_(k + n-l)) (79) 

which converts periodic parameters into an augmented time invariant vector. 

Analogously, we convert gi(k) into an augmented vector gj = [gi,i, gi,2> ■ ■ ■ r gi,N } an d 

meanwhile define a vector 

cpj{k) = [0, . . . , 0, 1, 0, . . . , 0] T G .R N «< (80) 

where the element 1 appears in the qth position of <Pj{k) only when k = sN„; + q. Hence 
for each time instance k, we have g;(k) = gj<Pj(k), i.e., gi(k) is converted into an augmented 
time-invariant vector. 
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Then, system (1) with periodic time-varying parameters 6j(k) and g;(k) can be transformed 
into 

n m 

V (k + n) = ^©J^ t (y(k + n-i)) + ^giVjWu(k-m+j) + v(z(k-T)) (81) 

,=\ ]=\ 

such that the method developed in Sections 4 and 5 is applicable to (81) for control design. 

9. Conclusion 

In this chapter, we have studied asymptotic tracking adaptive control of a general class of 
NARMA systems with both parametric and nonparametric model uncertainties. The effects 
of nonlinear nonparametric uncertainty, as well as of the unknown time delay, have been 
compensated for by using information of previous inputs and outputs. As the NARMA 
model involves future outputs, which bring difficulties into the control design, a future output 
prediction method has been proposed in Section 4, which makes sure that the prediction error 
grows with smaller order than the outputs. 

Combining the uncertainty compensation technique, the prediction method and adaptive 
control approach, a predictive adaptive control has been developed in Section 5 which 
guarantees stability and leads to asymptotic tracking performance. The techniques developed 
in this chapter provide a general control design framework for high order nonlinear 
discrete-time systems in NARMA form. In Sections 7 and 8, we have shown that the proposed 
control design method is also applicable to output-feedback systems and extendable to 
systems with periodic varying parameters. 
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1 1 . Appendix A: Proof of Proposition 3.1 

Only proofs of properties (ii) and (viii) are given below. Proofs of other properties are easy 
and are thus omitted here. 

(ii) From Definition 3.1, we can see that ||o[x(fc)] || < a(k) max/ l ./<; c+T ||x(/c') ||, Vfc > kg, r > 0, 
where \m\j c ^ O0 a(k) = 0. It implies that there exist constants k\ and Sj such that a (k) < a.\ < 1, 
Vfc > k\ . Then, we have 

||x(fc + T)+o[x(fc)]|| < ||x(fc + T)|| + ||o[x(fc)]|| < (1 + fti) max ||x(fc')||,Vfc > /q 

k'<k+T 

which leads to x(k + t) + o[x(fc)] = 0[x(k + t)]. On the other hand, we have 

max ||x(fc')ll < II max x(k') + o[x(k)}\\ + \\o[x(k)]\\ 
ki<k'<k+T k 1 <k'<k+r 

< || max *(0+o[x(fc)]||+i*i max {\\x(k')} 
ki<k'<k+T k!<k'<k+T 



and 



max \\x(k')\\ < 1| max x(k') + o\x(k')}\\yk > ki 

k!<k'<k+T 1-CCi k!<k'<k 
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which implies x(k + t) = 0[x(k) + o[x(k)]]. Then, it is obvious that x(k + t) + o[x(k)] ~ x(fc). 
(viii) First, let us suppose that X\{k) is unbounded and define z'/ c = argmax,<j- ||xx(i)||. Then, 
it is easy to see that i k — > oo as k — > oo. Due to limj.^ co a(fc) = 0, there exist a constant k 2 
such that a (fy) < J and ||o[xi(A:)]|| < i maxj/<j ||xi(fc')||, Vfc > ^2- Considering x 2 (k) = 

x\{k) + o[xi(k)], we have 

ll*2(»*)ll = WMk) + o[xi(i k )]\\ > \\xi(i k )\\ - \\o[xi(i k )]\\ > ^IM^II* Vfc > fc 2 

which leads to ||xi(ijt)|| < 2||x2(ijt)||, VA: > k 2 . Then, the unboundedness of x-y{k) conflicts 
with lim^co ||x2(fc)|| = 0. Therefore, X\ (k) must be bounded. Noting that a(k) — > 0, we have 

< \\ Xl (k)\\ < \\ Xl (k) +o[*i(fc)]|| + \\o[ Xl (k)]\\ < \\x 2 (k)\\ +«(fc)max||x 1 (fc')|| -► 

k'<k 

which implies lim^-s-oo II x l CO || = 0. 

12. Appendix B: Proof of Lemma 4.1 

It follows from (16) and (17) that 

y(k + l\k)=y(k + l\k)-y(k + l) 

n m 

= E ®h k ~ n + 2 ) A< fr( fc -»' + !) + E Sj( k - n + 2)Au(fc -m+j-n + 1) 
i=l ;=1 

-Av(ifc-n + l-T) (82) 

which results in 

n m 

-{ E S J( k ~ n + 2 ) A ^'( fc - * + !) + E 8j( k - n + 2)Au(k -m + j-n + l)}y(fc + l|fc) 
1=1 ;=1 

= -{y(k + l\k) + Ai/(fc - n + 1 - r)}y(fc + l|fc) 

= -y 2 (fc + l|fc)-At/(fc-n + l-T)y(fc + l|fc) 

< -y 2 (fc + l|fc) + A|y(fc + l|fc)|||AZ(fc-H + l)|| (83) 

To prove the boundedness of all the estimated parameters, let us choose the following 
Lyapunov function candidate 

k J n m \ 

M k ) = E E e 7(0 + E^ 2 ( 84 ) 

l=k-n+2 \i=\ ;=1 J 

Using the parameter update law (26), the difference of V p (k) is 

AV p (k) = V p (k+l)-V p (k) 

n m 

= £[ef(k+l)^9f(k-n + 2)] + £[gj(k + l)-gj(k-n + 2)] 
(=1 ,=i 

_ a 2 (k) 7 2 y 2 (k + l\k)irU ||Afr(fc - / + 1)|| 2 + E;=i Ah 2 (/c - m + ; - n + 1)] 2 a p (k) 7 

D 2 (k) D p (k) X 

n m 

{ £ §J(k -n + 2)A<pi(k - i + 1) + E |/(* - n + 2)Au(* - m +/ - n + l)}y(k + l\k) 

i=\ ;=1 
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According to the definition of D„(k) in (27) and inequality (83), the difference of V„(k) above 
can be written as 

at/ m s a2 P ( k h 2 f( k + 1 \ k ) 2a p (k) 7 f(k + l\k) 
A W - DJk) D^k) 



+ 



2a p (k)<y p A\y(k + l|fc)| || AZ(k -n + 1) 



D p (k) 
a 2 p (k)y 2 y 2 (k + l\k) 2a 2 p (khf(k + l\k) 



D p (k) D p (k) 



a 2 (k) 7p (2- 7p )y 2 (k+l\k) 



D p (fc) 
where the following equation obtained from the definition of dead zone (28) is used: 



(85) 



-2a 2 (khf(k + l\k) = -2a p (k) 7 y 2 (k + l\k) 

+2fl p (fc)7pA|y(fc + l|fc)|||AZ(fc -n + 1) || (86) 

Noting that < 7» < 2, we can see from (85) that the difference of Lyapunov function 
Vp(k), is non-positive and thus, the boundedness of V p (k) is guaranteed. It further implies 
the boundedness of Oj{k) and g;(k). Thus, there exist finite constants bg. and bg such that 

\\8i( k )\\<h,,gj(k)<b gj , Wkez+ n (87) 

Taking summation on both hand sides of (85), we obtain 

~ a 2 p (k)y(2- 7 )y 2 (k+l\k) 



/c=0 



D P (fc) 



Note that the left hand side of inequality (88) is the summation of a non-decreasing sequence 
and thus the boundedness of V p (k) implies 

al{k)f{k + l\k) 
' D p \k) - :-(*)->° W 

Noting that /jt-n+1 < fc — 2n + 1 by (7) and considering Assumption 3.1, (5) in Lemma 3.3, we 
see that D p (k) in (27) satisfies 

D)(fc)=0[y(fc + 1)] (90) 

From (89) and (90), we have 

a p (k)\g(k + l\k)\=ui(k)D*(k)=o[D$(k)]=o[0\y(k + l)]] (91) 

From the definition of dead zone in (28), when \y(k + l\k) \ > A|| AZ(k — n+ 1) |j, we have 

a p (k)\y{k + l\k)\ = |y(fc + l|fc)|-A||AZ(fc-n + l)|| >0 
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while when \y(k + l\k)\ < X£ v (k - n + 2)||AZ(fc- n + 1)||, we have 

a p (k)\y{k + l\k)\ = > \y(k + l\k)\ - A||AZ(fc-n + l)||. 
In summary, the definition of dead zone in (28) guarantees the following inequality 

\y(k + l\k)\ < a p {k) \y(k + l\k)\ + \c p {k-n + 2) ||AZ(fc - n + 1) || (92) 

which together with (91), boundedness of the parameter estimates, and the definition of 
A s (k,m) in (31) yields 

|y(fc + l|fc)|< [O[y(fc + l)]]+AciA s (fc,l) (93) 

with C\ = 1. Now, let us analyze the two-step prediction error: 

y(k + 2\k)=y(k + 2\k)-y(k + 2) 

=y(k + 2\k + l)+y(k + 2\k) (94) 

where 

y(k + 2\k + l) =y(k + 2\k + l)-y(k + 2) 

yik + 2\k) = y{k + 2|/c) - y(k + 2\k + 1) (95) 

From (93), it is easy to see that 

|y(fc + 2|fc + l)| < o[0[y(k + 2)]] + A Cl A s (fc,2) (96) 

From (17), and (20), it is clear that y(k + 2\k) in (95) can be written as 

y(k + 2\k)=y(k + 2\k)-y(k + 2\k + l) 

= §J(k-n + 3)[A$(k + l\k)-A<p(k + l)] (97) 

Using (93) and the Lipschitz condition of A</>,-(-) (or equivalently </>,(■)) with Lipschitz 
coefficient L,, we have 

||A#(fc+l|fc)-A«/>(fc + l)|| <L!|y(fc + l|fc)| KolOlyik+l^+te^Asik,!) (98) 

which yields 

\y{k + 2\k)\< [O[y{k+l)]]+\L 1 b 6l A s {^) (99) 

From (94), (96) and (99), it is clear that there exists a constant C2 such that 

|y(fc + 2|fc)| < 0[O[y(k + 2)]] + Ac 2 A s (fc,2) (100) 

Continuing the analysis above, for /-step estimate error y(k + l\k), we have 

y(k + l\k)=y(k + l\k)-y(k + l) 

= y(k + l\k)+y(k + l\k+l) (101) 

where 

y{k + l\k+l)=y{k + l\k+l)-y{k + l) 

y(k + l\k)=y{k + l\k)-y(k + l\k+l) (102) 
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For (7 — l)-step estimate error y(k + I — l\k), it can be seen that there exist constants Ci-\ and 
C;_l such that 

\y(k + l - l\k)\<o[0[y(k + l - 1)]] + Acj^A^M - 1) 

\y{k + I - l\k)\<o[0[y(k + l - 2)]] + Ac ! _ 1 A s (fc,Z - 2) (103) 

From (25) and (102), it is clear that y(k + l\k) can be expressed as 

/-l 
y(k + l\k) = £ §J(k -n + l + l)[A$(k + 1 - i\k) - A<f(fc + I - i\k + 1)] (104) 

i=l 

From (102), we have 

y(k + l-i\k)-$(k + l-i\k+l) =ij(k + l-i\k) (105) 

According to the Lipschitz condition of (p (■) and (105), the following equality holds: 

!-l \-\ 

£ || A0(* + / - i|fc) - A<f (fc + / - 1| < maxlLyh^n E l^(* + ' - »l*) I (106) 

i=i i=i 

From (93),(101)-(106), it follows that there exist constants C; such that 

|y(fc + Z|fc)|< [0[y(fc + Z)]] + AqA s (fc,Z) 
which completes the proof. 
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1. Introduction 

In this chapter, we report some work on decentralized adaptive control of discrete-time 
multi-agent systems. Multi-agent systems, one important class of models of the so-called 
complex systems, have received great attention since 1980s in many areas such as physics, 
biology, bionics, engineering, artificial intelligence, and so on. With the development of 
technologies, more and more complex control systems demand new theories to deal with 
challenging problems which do not exist in traditional single-plant control systems. 
The new challenges may be classified but not necessarily restricted in the following aspects: 

• The increasing number of connected plants (or subsystems) adds more complexity to the 
control of whole system. Generally speaking, it is very difficult or even impossible to 
control the whole system in the same way as controlling one single plant. 

• The couplings between plants interfere the evolution of states and outputs of each plant. 
That is to say, it is not possible to completely analyze each plant independently without 
considering other related plants. 

• The connected plants need to exchange information among one another, which may bring 
extra communication constraints and costs. Generally speaking, the information exchange 
only occurs among coupled plants, and each plant may only have local connections with 
other plants. 

• There may exist various uncertainties in the connected plants. The uncertainties may 
include unknown parameters, unknown couplings, unmodeled dynamics, and so on. 

To resolve the above issues, multi-agent system control has been investigated by many 
researchers. Applications of multi-agent system control include scheduling of automated 
highway systems, formation control of satellite clusters, and distributed optimization of 
multiple mobile robotic systems, etc. Several examples can be found in Burns (2000); Swaroop 
& Hedrick (1999). 

Various control strategies developed for multi-agent systems can be roughly assorted into 
two architectures: centralized and decentralized. In the decentralized control, local control 
for each agent is designed only using locally available information so it requires less 
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computational effort and is relatively more scalable with respect to the swarm size. In 
recent years, especially since the so-called Vicsek model was reported in Vicsek et al. (1995), 
decentralized control of multi-agent system has received much attention in the research 
community (e.g. Jadbabaie et al. (2003a); Moreau (2005)). In the (discrete-time) Vicsek model, 
there are n agents and all the agents move in the plane with the same speed but with different 
headings, which are updated by averaging the heading angles of neighor agents. By exploring 
matrix and graph properties, a theoretical explanation for the consensus behavior of the 
Vicsek model has been provided in Jadbabaie et al. (2003a). In Tanner & Christodoulakis 
(2005), a discrete-time multi-agent system model has been studied with fixed undirected 
topology and all the agents are assumed to transmit their state information in turn. In 
Xiao & Wang (2006), some sufficient conditions for the solvability of consensus problems 
for discrete-time multi-agent systems with switching topology and time-varying delays have 
been presented by using matrix theories. In Moreau (2005), a discrete-time network model 
of agents interacting via time-dependent communication links has been investigated. The 
result in Moreau (2005) has been extended to the case with time-varying delays by set-value 
Lyapunov theory in Angeli & Bliman (2006). Despite the fact that many researchers have 
focused on problems like consensus, synchronization, etc., we shall notice that the involved 
underlying dynamics in most existing models are essentially evolving with time in an 
invariant way determined by fixed parameters and system structure. This motivates us to 
consider decentralized adaptive control problems which essentially involve distributed agents 
with ability of adaptation and learning. Up to now, there are limited work on decentralized 
adaptive control for discrete-time multi-agent systems. 
The theoretical work in this chapter has the following motivations: 

1 . The research on the capability and limitation of the feedback mechanism (e.g. Ma (2008a;b); 
Xie & Guo (2000)) in recent years focuses on investigating how to identify the maximum 
capability of feedback mechanism in dealing with internal uncertainties of one single system. 

2. The decades of studies on traditional adaptive control (e.g. Astrom & Wittenmark (1989); 
Chen & Guo (1991); Goodwin & Sin (1984); Ioannou & Sun (1996)) focus on investigating 
how to identify the unknown parameters of a single plant, especially a linear system or 
linear-in-parameter system. 

3. The extensive studies on complex systems, especially the so-called complex adaptive systems 
theory Holland (1996), mainly focus on agent-based modeling and simulations rather than 
rigorous mathematical analysis. 

Motivated by the above issues, to investigate how to deal with coupling uncertainties as well 
as internal uncertainties, we try to consider decentralized adaptive control of multi-agent 
systems, which exhibit complexity characteristics such as parametric internal uncertainties, 
parametric coupling uncertainties, unmodeled dynamics, random noise, and communication 
limits. To facilitate mathematical study on adaptive control problems of complex systems, the 
following simple yet nontrivial theoretical framework is adopted in our theoretical study: 

1. The whole system consists of many dynamical agents, and evolution of each agent can be 
described by a state equation with optional output equation. Different agents may have 
different structures or parameters. 

2. The evolution of each agent may be interacted by other agents, which means that the 
dynamic equations of agents are coupled in general. Such interactions among agents 
are usually restricted in local range, and the extent or intensity of reaction can be 
parameterized. 
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3. There exist information limits for all of the agents: (a) Each agent does not have access 
to internal structure or parameters of other agents while it may have complete or limited 
knowledge to its own internal structure and values of internal parameters, (b) Each agent 
does not know the intensity of influence from others, (c) However, each agent can observe 
the states of neighbor agents besides its own state. 

4. Under the information limits above, each agent may utilize all of the information in hand 
to estimate the intensity of influence and to design local control so as to change the state of 
itself, consequently to influence neighbor agents. In other words, each agent is selfish and 
it aims to maximize its local benefits via minimizing the local tracking error. 

Within the above framework, we are to explore the answers to the following basic problem: 7s 
it possible for all of the agents to achieve a global goal based on the local information and local control? 
Here the global goal may refer to global stability, synchronization, consensus, or formation, 
etc. We shall start from a general model of discrete-time multi-agent system and discuss 
adaptive control design for several typical cases of this model. The ideas in this chapter can 
be also applied in more general or complex models, which may be considered in our future 
work and may involve more difficulties in the design and theoretical analysis of decentralized 
adaptive controller. 

The remainder of this chapter is organized as follows: first, problem formulation will be given 
in Section 2 with the description of the general discrete-time multi-agent system model and 
several cases of local tracking goals; then, for these various local tracking tasks, decentralized 
adaptive control problem for a stochastic synchronization problem is discussed in Section 3 
based on the recursive least-squares estimation algorithm; in Section 4, decentralized adaptive 
control for a special deterministic tracking problem, whereas the system has uncertain 
parameters, is given based on least-squares estimation algorithm; and Section 5 studies 
decentralized adaptive control for the special case of a hidden leader tracking problem, based 
on the normalized gradient estimation algorithm; finally, we give some concluding remarks 
in the last section. 

2. Problem formulation 

In this section, we will first describe the network of dynamic systems and then formulate 
the problems to be studied. We shall study a simple discrete-time dynamic network. In 
this model, there are N subsystems (plants), and each subsystem represents evolution of one 
agent. We denote the state of Agent i at time t by x,-(f), and, for simplicity, we assume that 
linear influences among agents exist in this model. For convenience, we define the concepts 
of "neighbor" and "neighborhood" as follows: Agent j is a neighbor of Agent ; if Agent j 
has influence on Agent i. Let Mi denote the set of all neighbors of Agent ; and Agent i 
itself. Obviously neighborhood Mi of Agent i is a concept describing the communication limits 
between Agent i and others. 

2.1 System model 

The general model of each agent has the following state equation (t = 1, 2, . . . , N) : 

Xi(t + 1) = f,(z,(t)) + u,(t) + 7i Xi{t) + iv,(t + 1) (2.1) 

with 2;(f) = [x i (t),u i (t)] T , x { {t) = [xj(t ), Xj(t - 1), . . . , Xj(t - n { + 1)] T and u t (t ) = 
[ui(t),Ui(t — 1), . . . ,M;(f — nii + 1)] , where /,(■) represents the internal structure of Agent 
i, Ui(t) is the local control of Agent i, u>i(t) is the unobservable random noise sequence, and 
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7,x,(f) reflects the influence of the other agents towards Agent i. Hereinafter, x;(£) is the 
weighted average of states of agents in the neighborhood of Agent i, i.e., 



Xi{t) = £ gijXj{t) 



(2.2) 



where the nonnegative constants {gij} satisfy V"' gij = 1 and 7, denotes the intensity of 

influence, which is unknown to Agent i. From graph theory, the network can be represented 
by a directed graph with each node representing an agent and the neighborhood of Node )' 
consists of all the nodes that are connected to Node / with an edge directing to Node i. This 
graph can be further represented by an adjacent matrix 

G = (g tj ), 8lj = 0i( j^M,. (2.3) 

Remark 2.1. Although model (2.1) is simple enough, it can capture all essential features that 
we want, and the simple model can be viewed as a prototype or approximation of more 
complex models. Model (2.1) highlights the difficulties in dealing with coupling uncertainties 
as well as other uncertainties by feedback control. 

2.2 Local tracking goals 

Due to the limitation in the communication among the agents, generally speaking, agents can 
only try to achieve local goals. We assume that the local tracking goal for Agent i is to follow a 
reference signal xj e , which can be a known sequence or a sequence relating to other agents as 
discussed below: 

Case I (deterministic tracking). In this case, x\ (t) is a sequence of deterministic signals 
(bounded or even unbounded) which satisfies |x' e (t) \ = 0(t ). 

Case II (center-oriented tracking). In this case, xj(t) = x(i) = i Y,j = i x i(t) is me center state 
of all agents, i.e., average of states of all agents. 

Case III (loose tracking). In this case, xj e (t) = Axj(t), where constant | A < 1. This case means 
that the tracking signal x* (t) is close to the (weighted) average of states of neighbor agents 
of Agent i, and factor A describes how close it is. 

Case IV (tight tracking). In this case, xj e (t) = x,(f). This case means that the tracking signal 
xf (t ) is exactly the (weighted) average of states of agents in the neighborhood of Agent i. 
In the first two cases, all agents track a common signal sequence, and the only differences are 
as follows: In Case I the common sequence has nothing with every agent's state; however, 
in Case II the common sequence is the center state of all of the agents. The first two cases 
mean that a common "leader" of all of agents exists, who can communicate with and send 
commands to all agents; however, the agents can only communicate with one another under 
certain information limits. In Cases III and IV, no common "leader " exists and all agents attempt 
to track the average state x,(f) of its neighbors, and the difference between them is just the 
factor of tracking tightness. 

2.3 Decentralized adaptive control problem 

In the framework above, Agent i does not know the intensity of influence 7,-; however, it can 
use the historical information 

{*,-(*), Xi(t), u,(t - l),x,-(f - l),x,(f-l),u,(f- 2),..., Xi(l),Xi(l),Ui(p)} (2-4) 
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to estimate 7, and can further try to design its local control u , ■ ( t ) to achieve its local goal. Such a 
problem is called a decentralized adaptive control problem since the agents must be smart enough 
so as to design a stabilizing adaptive control law, rather than to simply follow a common rule 
with fixed parameters such as the so-called consensus protocol, in a coupling network. Note that 
in the above problem formulation, besides the uncertain parameters 7,, other uncertainties 
and constraints are also allowed to exist in the model, which may add the difficulty of 
decentralized adaptive control problem. In this chapter, we will discuss several concrete 
examples of designing decentralized adaptive control laws, in which coupling uncertainties, 
external noise disturbance, internal parametric uncertainties, and even functional structure 
uncertainties may exist and be dealt with by the decentralized adaptive controllers. 

3. Decentralized synchronization with adaptive control 

Synchronization is a simple global behavior of agents, and it means that all agents tend to 
behave in the same way as time goes by. For example, two fine-tuned coupled oscillators 
may gradually follow almost the same pace and pattern. As a kind of common and important 
phenomenon in nature, synchronization has been extensively investigated or discussed in the 
literature (e.g., Time et al. (2004); Wu & Chua (1995); Zhan et al. (2003)) due to its usefulness 
(e.g. secure communication with chaos synchronization) or harm (e.g. passing a bridge 
resonantly). Lots of existing work on synchronization are conducted on chaos (e.g.Gade & 
Hu (2000)), coupled maps (e.g.Jalan & Amritkar (2003)), scale-free or small-world networks 
(e.g.Barahona & Pecora (2002)), and complex dynamical networks (e.g.Li & Chen (2003)), 
etc. In recent years, several synchronization-related topics (coordination, rendezvous, consensus, 
formation, etc.) have also become active in the research community (e.g.Cao et al. (2008); 
Jadbabaie et al. (2003b); Olfati-Saber et al. (2007)). As for adaptive synchronization, it has 
received the attention of a few researchers in recent years (e.g.Yao et al. (2006); Zhou et al. 
(2006)), and the existing work mainly focused on deterministic continuous-time systems, 
especially chaotic systems, by constructing certain update laws to deal with parametric 
uncertainties and applying classical Lyapunov stability theory to analyze corresponding 
closed-loop systems. 

In this section, we are to investigate a synchronization problem of a stochastic dynamic 
network. Due to the presence of random noise and unknown parametric coupling, 
unlike most existing work on synchronization, we need to introduce new concepts of 
synchronization and the decentralized learning (estimation) algorithm for studying the 
problem of decentralized adaptive synchronization. 

3.1 System model 

In this section, for simplicity, we assume that the internal function /,(■) is known to each 
agent and the agents are in a common noisy environment, i.e. the random noise {w(t), J-"t} 
are commonly present for all agents. Hence, the dynamics of Agent i (/ = 1, 2, . . . , N) has the 
following state equation: 

x,(t + 1) = fi(zi(t)) + tt,-(0 + yMt) + w(t + 1). (3.1) 

In this model, we emphasize that coupling uncertainty 7, is the main source to prevent 
the agents from achieving synchronization with ease. And the random noise makes that 
traditional analysis techniques for investigating synchronization of deterministic systems 
cannot be applied here because it is impossible to determine a fixed common orbit for 
all agents to track asymptotically. These difficulties make the rather simple model here 
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non-trivial for studying the synchronization property of the whole system, and we will find 
that proper estimation algorithms, which can be somewhat regarded as learning algorithms 
and make the agents smarter than those machinelike agents with fixed dynamics in previous 
studies, is critical for each agent to deal with these uncertainties. 

3.2 Local controller design 

As the intensity of influence 7, is unknown, Agent i is supposed to estimate it on-line via 
commonly-used recursive least-squares (RLS) algorithm and design its local control based on 
the intensity estimate 7;(£) via the certainty equivalence principle as follows: 

Ui (t) = -fi( Zi (t)) - Ut)m + xf(t) (3.2) 

where 7;(f ) is updated on-line by the following recursive LS algorithm 



7 ,(f + 1) = 7 ,(f) + Oi(t)Pi(t)Xi(t)[j/i{t + 1) - 7i(t)*i(t)] 
p l (t + l) = p i (t)-a l (t)[p i (t)x i (t)] 2 



with y { (t) = Xj(t) - fi(zi(t - 1)) - w,(f - 1) and 

-1 



*j(0 = [i + ft(0*?(*)] \pi(t) = 



(3.3) 



(3.4) 



Lfc=o 

Let ejj(t) = x,(t) - Xj(t), and suppose that xf i {t) = x*{t) for i = 1,2, . . . , N in Case I. And 
suppose also matrix G is an irreducible primitive matrix in Case IV, which means that all of 
the agents should be connected and matrix G is cyclic (or periodic from the point of view of 
Markov chain). 

Then we can establish almost surely convergence of the decentralized LS estimator and the 
global synchronization in Cases I — IV. 

3.3 Assumptions 

We need the following assumptions in our analysis: 

Assumption 3.1. The noise sequence {w (t ), Ft} is a martingale difference sequence (with { Ft) being 
a sequence of nondecreasing a-algebras) such that 

supE \\w(t+ 1)|^|^1 < 00 a.s. (3.5) 

for a constant /5 > 2. 

Assumption 3.2. Matrix G = (gif) is an irreducible primitive matrix. 

3.4 Main result 

Theorem 3.1. For system (3.1), suppose that Assumption 3.1 holds in Cases I — IV and Assumption 
3.2 holds also in Case IV. Then the decentralized LS-based adaptive controller has the following 
closed-loop properties: 
(1) All of the agents can asymptotically correctly estimate the intensity of influence from others, i.e., 

lim 7 ,(f) = 7,-. (3.6) 

f— >oo 
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(2) The system can achieve synchronization in sense of mean, i.e., 

lim ^£>y(OI=0, Sfi^j. (3.7) 

(3) The system can achieve synchronization in sense of mean squares, i.e., 

limi£k/«| 2 = °< ViV;. (3.8) 

3.5 Lemmas 

Lemma 3.1. Suppose that Assumption 3.1 holds in Cases I, II, III, and IV. Then, in either case, for 

i = 1,2, ... ,N and m > 1, < d < m, we have 



t 
y^ [■yi(mk — d)xj(mk — d)\ = o(t) a.s., 

Jt=l 

t 

Y\ \*fi{mk — d)xj(mk — d)\ = o(t) a.s. 
k=l 



(3.9) 



Proof. See Ma (2009). ■ 

Lemma 3.2. Consider the following iterative system: 

X t+1 = A t X, + W f , (3.10) 

where A t — > A as t — > oo and {Wf} satisfies 

Ell^H 2 = °( f )- (3-11) 

k=\ 

If the spectral radius p(A) < 1, then 

i,\\Xk\\=o(t), £||X ;c || 2 = o(f). (3.12) 

k=\ k=l 

Proof. See Ma (2009). ■ 

Lemma 3.3. The estimation 7;(f) of jj converges to the true value 7; almost surely with the 
convergence rate 



m 

where r,(f) and F,(f) are defined as follows 



-*=0 x i ( 

m = i+L' k -j xi(k) 



rM^l + L'fJ^m (3 . 14) 



Proof. This lemma is just the special one-dimensional case of (Guo, 1993, Theorem 6.3.1). 
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3.6 Proof of theorem 3.1 

Putting (3.2) into (3.1), we have 



Denote 



xi(t + 1) = -?j(*)*i(t) + xf{t) + jiXi(t) + w(t + 1) 
= xf(t)+7i(t)xi(t)+w(t + l). 

X(t) = (x 1 (t),x 2 (t),...,x N (t)) T , 
Z(t) = (xf(t),xf(t),...,xf(t)) T , 

X(t) = (x 1 (t),x 2 (t),...,x N (t)) T , 
W(t + 1) = w(t + 1)1 = (w(t + l),w(t + 1), . . . ,w(t + 1)) T , 
T(t) = dia S (y 1 (t),-y 2 (t),...,y N (t)), 

1=[1 1] T - 



(3.15) 



(3.16) 



Then we get 

X(t + 1) = Z(t) + T(t)X(t) + W(t + 1). (3.17) 

According to (2.2), we have 

X(t) = GX(t), (3.18) 

where the matrix G = (g;;). Furthermore, we have 

X(t + 1) = GX(t + 1) = GZ(t) + GT(t)X(t) + W(t + 1). (3.19) 

By Lemma 3.3, we have 7(f) — > as t — > 00. Thus, T(t) — > 0. 
By (3.15), we have 

Xj{t + l)-xf{t)-w{t + l) =-y,{t)xj{t). (3.20) 

Let ejj(t) = Xi(t) — Xj(t),rji(t) = 7,-(f)x,-(f). Then 

*ij(t + 1) = Mt) - t]j(t)] + [xf(t) - xf(t)]. (3.21) 

For convenience of later discussion, we introduce the following notations: 

G T =(d,C2,---,CN), 

E(t) = (e 1N (t),e 2N (t),...,e N _ hN (t),0) T , (3.22) 

ri(t) = (m(t),ri 2 (t),...,r lN (t)) T . 

Case I. In this case, xj (t) = x*(t), where x*(t) is a bounded deterministic signal. Hence, 

eij(t + l)=iji(t)-tij(t). (3.23) 

Consequently, by Lemma 3.1, we obtain that (i 7^ j) 

i:\e lj (k + l)\ 2 = o(£i 1 f(t))+o('tr I j(t)) =o(t), (3.24) 

k=l \k=\ ) \k=\ I 

and similarly YX=1 \ e ij(^ + 1) I = ° W a ^ so holds. 

f - 1 N 

Case II. In this case, xj e (t) = x(t) = jj J^ x i(tj- The proof is similar to Case I. 

1=1 
Case III. Here xf { (t) = Af,(f) = X(fX{t). Noting that £?1 = 1 for any i, we have 
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ZjX(t) - ZfX(t) = $[X(t) - x N (t)l] - tj[X(t) - x N (t)l] = CfE(t) - CJE(t), (3.25) 

and, thus, 

e,j(t + 1) = Mt) - 7/(01 + A fr'( f ) - *;(*)] 

= fo(0 - rij(t)\ + AKfx(f) - qX{t)] (3.26) 

= [in(t)-i ]j (t)} + \[cjE(t)-qE(t)]. 

Taking j = N and i = 1,2, ... ,N, we can rewrite (3.26) into matrix form as 

E(t + 1) = [rj{t) - m {t)l] + \[G - Hl\E(t) = AHE(t) + f (f), (3.27) 

where 



H=G _ Gn = G - lf T / ?(f) = 7(t) _ ^(^ (3 28) 



By Lemma 3.1, we have 



5ZH'7(*)ll 2 = o(*). (3-29) 



/c=l 

Therefore, 

t,U(k)\\ 2 = o(t). (3.30) 

k=l 
Now we prove that p(H) < 1. In fact, for any vector p such that c T o = 1, we have 

w r Hc> = |u T Gu — v T G}jv\ 

< max (A max (G)||u|| 2 - A min (G N )||u|| 2 , 
A ma x(G N )|M| 2 -A min (G)|M| 2 ) (3.31) 

< max (||»|| 2 ,A m ax(G N )||i;|| 2 ) 
= 1 

which implies that p(H) < 1. 

Finally, by (3.27), together with Lemma 3.2, we can immediately obtain 

£||E(k)|| =o(t), Ell £ WH 2 = °( f )- (3-32) 

it=i jt=i 

Thus, for i = 1, 2, . . . , N — 1, as t — ¥ oo, we have proved 

yEl^WI^O, y X>n(?<)] 2 ^ 0. (3.33) 

/c=l Jc=l 

Case IV. The proof is similar to that for Case III. We need only prove that the spectral radius 
p(H) of H is less than 1, i.e., p(H) < 1; then we can apply Lemma 3.2 like in Case III. 
Consider the following linear system: 

z(f + l) = Gz(f). (3.34) 

Noting that G is a stochastic matrix, then, by Assumption 3.2 and knowledge of the Markov 
chain, we have 

lim G f = l7T r , (3.35) 

f— >oo 
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where n is the unique stationary probability distribution of the finite-state Markov chain with 
transmission probability matrix G. Therefore, 

z(f) = G f z(0) -> ln T xf = {K T xf)l (3.36) 

which means that all elements of z(f) converge to a same constant n x\ . Furthermore, let 
2 (f) = (xf(t),xf(t),...,x^{t)) T andv(f) = (vi(0,vs(0/- • .,i/ N -iW,0) r , where v,{t) = 
xf s (t) - X T § f (t) for i = 1, 2, . . . , N. Then we can see that 

v(t + l) = (G-G N )v{t) = Hv(t) (3.37) 

and limf_ s . 00 i/(f) = for any initial values v,-(0) G 7^, i = 1,2, ...,N — 1. Obviously v(f) = 
H f i/(0), and each entry in the Nth row of H is zero since each entry in the Nth row of H is 
zero. Thus, denote 

'H (t) * 
' 



H^ 



(3.38) 



where H (f) is an (N - 1) x (N - 1) matrix. Then, fori = 1,2,. . .,N - 1, taking i/(0) = e„ 
respectively, by lim v(t) = we ea 

as t — > oo. Consequently, we have 



respectively, by lim v(t) = we easily know that the !th column of Hg(i) tends to zero vector 

t— >oo 



limH (f)=0, (3.39) 

t— >oo 

which implies that each eigenvalue of Ho(f) tends to zero too. By (3.38), eigenvalues of H l are 
identical with those of Ho(f) except for zero, and, thus, we obtain that 

lim p (H 1 ) = (3.40) 

which implies that 

p(H) < 1. (3.41) 

This completes the proof of Theorem 3.1. ■ 

4. Decentralized tracking with adaptive control 

Decentralized tracking problem is critical to understand the fundamental relationship 
between (local) stability of individual agents and the global stability of the whole system, 
and tracking problem is the basis for investigating more general or complex problems such as 
formation control. In this section, besides the parametric coupling uncertainties and external 
random noise, parametric internal uncertainties are also present for each agent, which require 
each agent to do more estimation work so as to deal with all these uncertainties. If each agent 
needs to deal with both parametric and non-parametric uncertainties, the agents should adopt 
more complex and smart leaning algorithms, whose ideas may be partially borrowed from Ma 
& Lum (2008); Ma et al. (2007a); Yang et al. (2009) and the references therein. 

4.1 System model 

In this section, we study the case where the internal dynamics function /,•(■) is not completely 
known but can be expressed into a linear combination with unknown coefficients, such that 
(2.1) can be expressed as follows: 

rij nij 

Xi(t + 1) + E a ik Xi(t - k + 1) = E b lk u t (t - k + 1) + 7; £ g lj xAt) + w,(t + 1) (4.1) 
Jt=i Jt=i jeM, 
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which can be rewritten into the well-known ARMAX model with additional coupling item 
E/W, gij x j(t) (letting g t j = 7^) as follows: 

A l {q- l )x l {t + 1) = Bi{q- l )Ui(t) + v>i{t + 1) + E gijXj(t) (4 2 ) 

jeNj ' v ' ' 

with A t {q- 1 ) = 1 + Yl]L\ a ifi~ ] i ^(T 1 ) = h il + L?=2 h ij c l' j+l and back shifter tf l . 

4.2 Local controller design 

For Agent i, we can rewrite its dynamic model as the following regression model 

Xi(t + 1) = ef&it) + Wi(t + 1) (4.3) 

where 0, holds all unknown parameters and </>,(() is the corresponding regressor vector. 
Then, by the following LS algorithm 

§i(t + i) = 0^(0 + ^0^(0^0 [*,(f + i)-<^(O0,(O] 

Pi(t + 1) = p,(t) - ^(0^(0^(0^(0^(0 (4-4) 

£7,(0 =[i + 4>J{t)P i (t)4> i (t)]- 1 

we can obtain the estimated values 0,(0 of 0, at time t. For Agent i, to track a given local 

reference signal x, (t) = x*(t), with the parameter estimate 0,'(O given by the above LS 
algorithm, it can then design its adaptive control law m,(0 by the "certainty equivalence" 
principle, that is to say, it can choose «;(0 such that 

6j(t)4> l (t) = x*(t + l) (4.5) 

where x*(t) is the bounded desired reference signal of Agent i, i.e. Agent i is to track the 
deterministic given signal x*(t). 
Consequently we obtain 

u l (t) = ^ f) {x*(t + l) 

+ [a il {t)x i {t) + ■■■ + a ipi {t)xj{t - pi + 1)] , 4 6) 

-[b t2 (t)u t (t - 1) + • • • + b li(ji (t)u,(t - q, + 1)] 

-9l(t)X,(t)} 

where g, ■ ( is a vector holding the estimates §n(t) of g, ■ ,■ (7 € A/";) and Xj(t) is a vector holding 

the states x t j(t) (j e A/ - ,-). 

In particular, when the high-frequency gain b{\ is known fl priori, let 0, denote the parameter 

vector 0, without component fc,j, 0/(0 denote the regression vector </>,(0 without component 

«,(0/ and similarly we introduce notations «,-(0/jP;(0 corresponding to S;(0 and P,(0/ 

respectively. Then, the estimate 0/(0 a t time t of 0; can be updated by the following 

algorithm: 

di(t + i) = 0,(0 + ^(0^(0^(0 

x[x t (t + l)-b, lUl (t)-4>J(t)0,(t)} 

P,(t + 1) = p,(0 - ti(t)Pi(t)fo(t)$i(t)Pi(t) ' 

a.it) =[l + 4>J(t)P i (t)4> l (t)]-i 
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When the high-frequency gain h{\ is unknown a priori, to avoid the so-called singularity 
problem of bn(t) being or approaching zero, we need to use the following modified bn(t), 

denoted by b,i(f), instead of original fo;i(t): 

(Mf) ifiBaCOI^-^jj 

and consequently the local controller of Agent i is given by 

U,(t) = 1 {rf(f+l) 
MO ' 



+ [an(t)x,-(0 + ■ ■ ■ + a iiVi (t)xi(t - Pi + 1)] 
-[6 a (0«i(t -!) + ••• + k, qi (t)Ui(t - q, + 1)] 

-gf(t)x,(t)}. 



(4.9) 



4.3 Assumptions 

Assumption 4.1. (noise condition) {wj(t), Ft} is a martingale difference sequence, with {Ft} being 
a sequence of nondecreasing a-algebras, such that 

supE[\wi(t + l)\P\F t } < oo,a.s. 

f>0 

for some f> > 2 and 

lim I £ K(*)| 2 = Rj > 0,a.s. 
f^oo , c=1 

Assumption 4.2. (minimum phase condition) Bj(z) ^ 0,Vz S C : |z| < 1. 

Assumption 4.3. (reference signal) {x* (t) } is a bounded deterministic signal. 

4.4 Main result 

Theorem 4.1. Suppose that Assumptions 4.1 — 4.3 hold for system (4.1). Then the closed-loop system 
is stable and optimal, that is to say, for i = 1,2, . . . ,N, we have 

limsupj E [\xi{k)\ 2 + \iii(k-l)\ 2 } < oo, a.s. 

and 

limi £ \ Xl (k)-x*(k)\ 2 = R„ as. 

f-S-oo k=l 

Although each agent only aims to track a local reference signal by local adaptive controller 
based on recursive LS algorithm, the whole system achieves global stability. The optimality 
can also be understood intuitively because in the presence of noise, even when all the 
parameters are known, the limit of 



f-1 

2 



7;(0 = Hm 7 E |x,-(k + l) -*?(*+ 1) 

f^oo k=Q 



cannot be smaller than Rj. 
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4.5 Lemmas 

Lemma 4.1. Under Assumption 4.1, we have \wj(t)\ = 0(d,-(£)), where {d,(f)} is an increasing 

sequence and can be taken as t s (S can be any positive number). 

Proof. In fact, by using Markov inequality, we obtain that 

[=1 f=l 

holds almost surely. By applying the Borel-Cantelli-Levy lemma, immediately we have 

K(£ + l)| =0{t s ),a.s. m 

Lemma 4.2. If£(t + 1) = B(z)u{t),\/t > 0, where polynomial (q > 1) 

B(z) = b 1 + b 2 z+--- + b qZ 1- 1 

satisfies 

B(z) ^0,Vz: \z\ < 1, (4.10) 

then there exists a constant A £ (0, 1) such that 

/ + ! 



Kt)| 2 = o(EA t+1 -^(fc)| 2 ). (411) 

k=0 



Proof. See Ma et al. (2007b). 



Lemma 4.3. Under Assumption 4.1, for i = 1,2, ... ,N, the LS algorithm has the following properties 

almost surely: 

(a) 

6f(t + l)Pr\t + l)0i(t + 1) = 0(logr,(f)) 
(b) 

E a,(k) = 0(\ogr,(t)) 

Jc=l 

where 

di(t) ±tr(Pi(t)-Pi(t + l)) 

*&) ± [l + 4>J(k)Pt(k)4>i{k)]-* 

«,(fc) = ^-(fc)|0, T (fc)^WI 2 (412) 

r { (t) =1+ L<t>J(k)4n(k) 



k=l 
Proof. This is a special case of (Guo, 1994, Lemma 2.5). ■ 

Lemma 4.4. Under Assumption 4.1, for i = 1,2, ... ,N, we have 

E \xi(k)\ 2 ->■ oo, liminf \ E \x,(k)\ 2 > R, > 0, a.s. (4.13) 

k=\ ^°° k=\ 

Proof. This lemma can be obtained by estimating lower bound of E [ x i(k + 1)] with the help 

k=\ 
of Assumption 4.1 and the martingale estimation theorem. Similar proof can be found in Chen 

& Guo (1991). ■ 



242 Discrete Time Systems 

4.6 Proof of Theorem 4.1 

To prove Theorem 4.1, we shall apply the main idea, utilized in Chen & Guo (1991) and Guo 
(1993), to estimate the bounds of signals by analyzing some linear inequalities. However, there 
are some difficulties in analyzing the closed-loop system of decentralized adaptive control 
law. Noting that each agent only uses local estimate algorithm and control law, but the agents 
are coupled, therefore for a fixed Agent ;', we cannot estimate the bounds of state X{(t) and 
control Uj(t) without knowing the corresponding bounds for its neighborhood agents. This is 
the main difficulty of this problem. To resolve this problem, we first analyze every agent, and 
then consider their relationship globally, finally the estimation of state bounds for each agent 
can be obtained through both the local and global analysis. 
In the following analysis, <5,-(f), c,-(fc), tx.j(k) and r,-(f) are defined as in Eq. (4.12). 
Step 1: In this step, we analyze dynamics of each agent. We consider Agent ; for i = 
1,2, ... ,N. By putting the control law (4.9) into (4.3), noting that (4.5), we have 

Xj(t + i) = ef4>i(t) + wi(t+i) 

= x* (f + 1) - 6T{t)Mt) + oj4>i{t) + io.it + 1) 
= x* {t + 1) + SJ (t^iit) + ivi{t + 1) 

By Lemma 4.1, we have \wj(t ) 2 = 0(d,(f)). Noticing also 

|0,(f)^(f)| 2 = «,(f)[l + ^ r (f)P,(f)^(f)] 

+CLi(t)<f»J{t)[Pi(t)-P t (t + l))4>i(t) 

<«,-(0[2 + 4(0ll«fc(0ll 2 ] 

and the boundedness of x* (t + 1), we can obtain that 

|*i(*+l)l 2 < 2 a; (0^(f)ll^(f)l| 2 + O(d,(f)) + O(logr ; (f)). (4.14) 

Now let us estimate ||</>;(f) || • By Lemma 4.2, there exists A,- g (0, 1) such that 

|"«(f)| 2 = 0(E 1 A| +1 - ,: (|x ; (fc)| 2 +||X,(fc)|| 2 + | t( ;,(fc + l)| 2 ))- 

J:=0 

It holds for all i = 1,2, ... ,N, but we cannot estimate |w,(£) | 2 directly because it involves 

{*,-(*),/ eJV,} in **(*)• 
Let 

p = max(Ai, ■ ■ ■ , A N ) G (0, 1) 

X(k) = { Xl (k),---,x N (k)] T 
d(k) = max(rfi(fc), ■ ■ • ,djv(fc)). 

Obviously we have 

|x,(fc)| 2 = 0(||X(fc)|| 2 ),||XKfc)|| 2 = 0(||X(fc)|| 2 ). 
Now define 



A 






u= i: P <- k \\x{k)\ 

k=0 



Decentralized Adaptive Control of Discrete-Time Multi-Agent Systems 243 

Then, for i = 1,2, ... ,N, we have 

\u t (t)\ 2 = 0(L t+1 ) + 0(E P t+1 - k d(k)) 

= 0(L t+1 ) + 0(d(t + l)). 

Since 

4>i(t) = [xi(t), ■ ■ ■ , Xi (t - Pi + 1), Ui(t - 1), ■ ■ ■ , Ui (t - m + l),XT(t)] T 

we can obtain that 

\\4>,(t)\\ 2 = 0(\\X(t)\\ 2 ) + 0(L t )+0(d(t)) 
+0(lo S r,(t) + di(t)) 
= 0{L t + logT{t)+d(t)) 

where 

f(t) = max(r 1 (t),f Z (f),---,r N (0). 
Hence by (4.14), for Agent i, there exists C; > such that 

|x ; (f + l)| 2 < Cia.i(t)6i(t)Lt 

+0(a i (t)6 i (t)[\ogr(t)+d(t)}) 
+0(d,(t) +\ogr,(t)). 

Then noticing 

oi(t)Si{t) = oQognW) 

we obtain that 

|x ; (f + l)| 2 < Q« l -(f)<5i(i)L t + 0(logr i (f)[logf(f) + i(f)])- (4-15) 

Step 2: Because (4.15) holds for i = 1,2, . . . ,N, we have 

l|x(t + i)|| 2 = E|n(t + i)| 2 

1=1 

N 

< [L Cm(t)di(t)]L t 

+0(Nd(f)logr(f))+0(Nlog 2 F(f)). 
Thus by the definition of Lt, we have 

L m =pL ( +||X(f + l)|| 2 

N 

+0{Nd(t) logf (£)) + 0(Nlog 2 f(t)) 

where 

C = max(Ci,C2, • • ■ ,Cjv). 

N 

Let 77(f) = £ «,-(f)<5,-(f),fhen 

1=1 

L t+l = 0(Nd(t) logr(t) + N log 2 f(t)) 

+0{NZ n (p + Ci|(/)] (4.16) 

x[d(fc)logf(fc)+log 2 f(fc)]). 
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Since 



E Si(k) = E [trP,-(fc) - trP,-(k + l)] < trP,(0) < oo, 

S:=0 k=0 



we have Sj(k) — > as k — > oo. By Lemma 4.3, 

CO 

E «,-(*) =0(Iogr,-(fc))=0(logf(*)). 

k=a 

Then, for i = 1,2, . . . ,N and arbitrary e > 0, there exists kg > such that 

p- l C E *i(k)6i(k) < i^elogr(t) 
fc=i 

for all t > to > kg. Therefore 

p~ l C E r,{k) < elogf(f). 

fc=f 

Then, by the inequality 1 + x < e x , Vx > we have 

A [l + p - x Crj(k)] < explp-iC E r,(k)} 

k=h k=t 

< exp{elogf(£)} = f £ (t). 

Putting this into (4.16), we can obtain 

L f+1 = 0(logr^)[logr(f) + d(f)]f e (f)). 
Then, by the arbitrariness of e, we have 

L m = O(d(t)r £ {t)),\/e>0. 
Consequently, for i = 1, 2, . . . , N, we obtain that 

\\X(t + l)\\ 2 <L t+1 = 0(d(t)f £ (t)) 

\u i (t)\ 2 = 0(L t+1 + d(t + l))=0(d(ty(t)) (4.17) 

U,(t)\\ 2 = 0(L t + \o S r(t) + d(t)) = 0(d(t)r e (t)). 
Step 3: By Lemma 4.4, we have 

liminf 4^ > Ri > 0, a.s. 

Thus f = 0(r,(f)) = 0(F(f)), together with d(f) = O^VJ G (|,1), then we conclude that 
d(t) = 0(f e (t)). Putting this into (4.17), and by the arbitrariness of e, we obtain that 



||<A,(f)l[ 2 = 0(f*(f)),WG(f,l). 
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Therefore 



E \dj(k)4>,(k)\ 2 

k=0 

= L *i{k)[l + 4>J(k)Pi(k)4H(Q] 

k=0 

= 0(logr,(f)) + 0(E*,W !!</>,« II 2 ) 

k=0 
= 0(\ogr(t))+0(r s (t) Z*i(k)) 

= 0(f' 5 (f)logf(f)) / V<5g[l,l). 



Then, by the arbitrariness of 5, we have 



t 



Since 
we have 



E \B (fc)^Wr = 0(r*(t)),\/6 e (4,1). (4.18) 

k=0 F 



Xi(t + 1) = OHt)(f>i(t) + xUt + l)+Wi(t + l) 



E \x t (k + 1)| 2 = 0{T s {t)) + 0(f) + 0(logf(f)) 
= 0(r d '(f)) + 0(f) 

E K(fc-i)| 2 = o(r*(t)) + o(0 

J:=0 



From the above, we know that for i = 1,2, . . . ,N, 

n(t) = i+ e \\4> t (k)\\ 2 = o(? s (t)) + o(t) 

k=0 

vse(j,i). 

Hence 

T(t) = max{r,(f),l < i < N} 

= 0(r s (t)) + 0(t),V6e(j,l). 

Furthermore, we can obtain 

r(t)=0(t) 

which means that the closed-loop system is stable. 
Step 4: Now we give the proof of the optimality. 

E \ Xi (k + l)-x?(k + l)\ 2 

k=0 

= EWHi)f+ E[fc(*)] 2 + 2 E ipi(k)wi(k + l) 

k=0 k=0 k=0 

where 

ip^^ejik^ik)- 



(4.19) 
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By (4.18) and the martingale estimate theorem, we can obtain that the orders of last two items 
in (4.19) are both 0(f s (t)),\/S e (1,1)- Then we can obtain 

limf E \x,{k + l)-x*{k + l)\ 2 = R i , a.s. 



Furthermore 



E \x t (k)-x*(k)-w,(k)\ 2 = e ||fc(*)| 

k=0 k=0 



= 0{r{t))=o{t), a.s. 
This completes the proof of the optimality of the decentralized adaptive controller. ■ 

5. Hidden leader following with adaptive control 

In this section, we consider a hidden leader following problem, in which the leader agent 
knows the target trajectory to follow but the leadership of itself is unknown to all the others, 
and the leader can only affect its neighbors who can sense its outputs. In fact, this sort of 
problems may be found in many real applications. For example, a capper in the casino lures 
the players to follow his action but at the same time he has to keep not recognized. For another 
example, the plainclothes policeman can handle the crowd guide work very well in a crowd 
of people although he may only affect people around him. The objective of hidden leader 
following problem for the multi-agent system is to make each agent eventually follow the 
hidden leader such that the whole system is in order. It is obvious that the hidden leader 
following problem is more complicated than the conventional leader following problem and 
investigations of this problem are of significance in both theory and practice. 

5.1 System model 

For simplicity, we do not consider random noise in this section. The dynamics of the 
multi-agent system under study is in the following manner: 

Aiiq-^Xiit + l) = B i {q- 1 )u i {t)+<y i x i {t) (5.1) 

with Ai{q~ l ) = 1 + E!=! a-ifl~K Bi(q^) = b,\ + Ej=2 fo 'yT y+1 and back shifter q~ l , where 
Uj(t) and X{(t), i = 1,2, ... ,N, are input and output of Agent i, respectively. Here X{(t) is the 
average of the outputs from the neighbors of Agent i: 

*/(*) = W E Xj(t) (5.2) 

/GjV, 

where 

M = {s;,i,s,', 2 ,...,si,ivj (5.3) 

denotes the indices of Agent i's neighbors (excluding Agent i itself) and N, is the number of 
Agent i's neighbors. In this model, we suppose that the parameters a,-,- (/ = 1, 2, ... , mi), b{j 
(j = 1, 2, ... , n ;) and 7; are all a priori unknown to Agent i. 

Remark 5.1. From (5.1) we can find that there is no information to indicate which agent is the leader 
in the system representation. 
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5.2 Local Controller design 

Dynamics equation (5.1) for Agent i can be rewritten into the following regression form 

x i (t + i) = ef^ i (t) 

where 0, holds all unknown parameters and </>,(() is the corresponding regressor vector. 
We assume that the bounded desired reference x*(k) is only available to the hidden leader 
and satisfies x*(k + 1) — x*(k) = o(l). Without loss of generality, we suppose that the first 
agent is the hidden leader, so the control U\(t) for the first agent can be directly designed by 

using the certainty equivalence principle to track x T f (k) = x* (k): 

ej(t)4> 1 (t) = x*(t + i) (5.4) 

which leads to 

u\if) = ^KO + 1) + {h\{t)xi{t) + ■■■ + a ljh (f)*i(f - n x + 1)] 

-[&12(0«l(* -!) + ••■ + km (0«l(* - m i + 1)1 (5 - 5) 

-7i(t)xi(t)}. 

As for the other agents, they are unaware of either the reference trajectory or the existence 
of the leader and the outputs of their neighbors are the only external information available 
for them, consequently, the jih (j = 2, 3, ■ ■ ■ , N) agent should design its control uAt) to track 

corresponding local center x'j (t) =Xs(t) such that 

6J(t)<t>j(t) = Xj(t) (5.6) 

from which we can obtain the following local adaptive controller for Agent j: 

M /W = b^ij^/ (f) + M f ) x /W + ■ ■ ■ + W f )* ; (f - n i + !)] 

-[b j2 (t)uj(t - 1) + • • • + b j>mj (t)uj(t - mj + 1)] (5-7) 

-ij(t) Xj (t)}. 



Define 



and 



y 1 {t) = x l {t)-x*{t) (5.8) 



yj{t) = Xj(t)-Xj(t-1), j = 2,3,---,N. (5.9) 

The update law for the estimated parameters in the adaptive control laws (5.5) and (5.7) is 
given below (j = 1,2,..., N): 

6j(t) -o,(t-i)+ Dj(t _ r) 

D j (k) = l+\\<p j (k)\\ 2 

where < ]l; < 2 is a tunable parameter for tuning the convergence rate. Note that the above 
update law may not guarantee that bMt ) > bn, hence when the original b.-j ( t ) given by (5.10), 
denoted by V-At) hereinafter, is smaller than b», we need to make minor modification to on (t) 
as follows: 

h jl (t) = b jl if i' n (t) < b n . (5.ii) 

In other words, b;i(f) = max(B' 1 (f),fo. 1 ) in all cases. 
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5.3 Assumptions 

Assumption 5.1. The desired reference x* (k) for the multi-agent system is a bounded sequence and 
satisfies x*(k + 1) - x*(k) = o(l). 

Assumption 5.2. The graph of the multi-agent system under study is strongly connected such that 
its adjacent matrix Ga is irreducible. 

Assumption 5.3. Without loss of generality, it is assumed that the first agent is a hidden leader who 
knows the desired reference x* (k) while other agents are unaware of either the desired reference or which 
agent is the leader. 

Assumption 5.4. The sign of control gain bn, 1 < / < n, is known and satisfies \b;\\ > b ;1 > 0. 
Without loss of generality, it is assumed that bn is positive. 

5.4 Main result 

Under the proposed decentralized adaptive control, the control performance for the 
multi-agent system is summarized as the following theorem. 

Theorem 5.1. Considering the closed-loop multi-agent system consisting of open loop system in (5.1) 
under Assumptions 5.1-5.4, adaptive control inputs defined in (5.5) and (5.7), parameter estimates 
update law in (5.10), the system can achieve synchronization and every agent can asymptotically track 
the reference x* (t ), i.e., 

lime(t) = 0, / = 1,2,...,N (5 . 12 ) 

where eAk) = x;(k) — x*(k). 

Corollary 5.1. Under conditions of Theorem 5.1, the system can achieve synchronization in sense of 
mean and every agent can successfully track the reference x*(t) in sense of mean, i.e., 

lim I E \e;(k)\=0, i = l,2,...,N (5.13) 

£-+«> , c=1 i 

5.5 Notations and lemmas 

Define 

X(k) = [ Xl (k),x 2 (k),...,x N (k)} T (5.14) 

Y(k) = [y,(k),y 2 (k),...,y n (k)} T (5.15) 

H=[1,0,...,0] T elZ N (5.16) 



From (5.2) and (5.14), we have 




[0,x 2 (k),. 


..,*»(*)]=/ 


where 


"0 ■■■ 


A = 


«&••• ° 




_o o ...^ 



(5.17) 



(5.18) 
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and Ga is an adjacent matrix of the multi-agent system (5.1), whose (i,/)th entry is 1 if / € 
M{ or if / £ Mi- Consequently, the closed-loop multi-agent system can be written in the 
following compact form by using equality 

X(k + 1) = AG A X(k) + Hx*(k + 1) + Y(k + 1) (5.19) 

Definition 5.1. A sub-stochastic matrix is a square matrix each of whose rows consists of nonnegative 
real numbers, with at least one row summing strictly less than 1 and other rows summing to 1. 

Lemma 5.1. According to Assumption 5.2, the product matrix AGj\ is a substochastic matrix (refer 
to Definition 5.1) such that p(AGj\) < 1 (Dong et al. (2008)), where p(A) stands for the spectral 
radius of a matrix A. 

Definition 5.2. (Chen & Narendra, 2001) Let X\ (k) and xi (k) be two discrete-time scalar or vector 
signals, \/k G Z^,for any t. 

• We denote x\ (k) = 0[x2(k)], if there exist positive constants m\, mi and kg such that \\x\ (k) \\ < 
mimax t /< t ||x 2 (fc')ll + m 2, Vfc > k . 

• We denote x\{k) = o[x2{k)\, if there exists a discrete-time function a(k) satisfying \imi c ^ BO a(k) = 
and a constant kg such that \\xi(k)\\ < a(k) max yt / <yt ||x2(fc')||, Vfc > kg. 

• We denote xi(k) ~ X2(k) if they satisfy x\(k) = 0[x2{k)] and X2(k) = 0[x\(k)\. 

For convenience, in the followings we use 0[1] and o[l] to denote bounded sequences and 
sequences converging to zero, respectively. In addition, if sequence y(k) satisfies y(k) = 
0[x(k)] or y(k) = o[x(k)], then we may directly use 0[x(fc)] or o[x(/c)] to denote sequence 
y(k) for convenience. 
According to Definition 5.2, we have the following lemma 

Lemma 5.2. According to the definition on signal orders in Definition 5.2, we have following 
properties: 

(i) 0[xi(fc+T)]+0[*l(fc)] ~O[xi(fc+T)],VT>0. 

(ii) Xl {k + T) + o [ Xl (fc)] ~ x 1 (k + t), Vt > 0. 

(Hi) o[xi(k + r)] + o[xi(k)] ~ o[xi(k + r)], Vt > 0. 

(to) o[ Xl (k)}+o[x 2 (k)} ~ o[|xi(fc)| + \x 2 (k)\}. 

(v) o[0[x 1 (k)]]~o[x 1 (k)] + 0[l]. 

(vi) lfxi(k) ~ x 2 (k) andlimj-^co ||x 2 (fc)|| = 0, then lim^oo ||xi(fc)|| = 0. 

(vii) Ifx\(k) = o\x\(k)\ + o[l], then lim/t^oo ll x i(fc)ll = 0. 

(mi) Letx 2 (k) = x 1 (k) + o[x 1 (k)]. Ifx 2 (k) = o[l},then\hn k ^ 00 \\x 1 (k)\\ =0. 

The following lemma is a special case of Lemma 4.4 in Ma (2009). 

Lemma 5.3. Consider the following iterative system 

X{k + 1) = A(k)X(k) + W(k) (5.20) 

where \\W(k) || = 0[1], and A(k) — > A as k — > oo. Assume that p(A) is the spectral radius of A, i.e. 
p{A) = max{ | A( A) \ } and p(A) < 1, then we can obtain 

X(fc + l) = 0[l]. (5.21) 
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5.6 Proof of Theorem 5.1 

In the following, the proof of mathematic rigor is presented in two steps. In the first step, we 
prove that £j(k) -¥ for all ;' = 1, 2, . . . , N, which leads to %\ (k) - x* (k) -* such that the 
hidden leader follows the reference trajectory. In the second step, we further prove that the 
output of each agent can track the output of the hidden leader such that the control objective 
is achieved. 
Step 1: Denote 5j(k) =6j{k) — 0;(k), especially bj\(k) = bn{k) — b;\. For convenience, let 

ft = &ji — bji, where oL denotes the original estimate of b,\ without further modification. 
From the definitions of b',-. and bn, since bn{i) = max(bL(t),bji) and b» > b_n, obviously we 
have 

Consider a Lyapunov candidate 



P n (k) < b' 2 (k). (5.22) 



Vj(k) = \\ej(k)\\ 2 (5.23) 

and we are to show that V;(k) is non-increasing for each j = 1,2, ... ,N, i.e. V; (k) < Vi(k—1). 
Noticing the fact given in (5.22), we can see that the minor modification given in (5.11) will not 
increase the value of V;(k) when bL(k) < bj\, therefore, in the sequel, we need only consider 
the original estimates without modification. Noting that 

||fy(fc) - §j(k - 1)|| = ||e ; (/c) - 9j(k - 1)|| (5.24) 

the difference of Lyapunov function Vi(k) can be written as 

AVj(k) =Vj(k)-Vj(k-l) 

= \\e j (k)f-\\e j (k-i)\\ 2 

= \\§j(k) - §j(k- l)\\ 2 + 28j{k- l)[§j{k) - 6j(k-l)]. 

(5.25) 

Then, according to the update law (5.10), the error dynamics (5.8) and (5.9), we have 

\\§j(k) - §j(k -l)\\ 2 + 26](k-l) [§j(k) - 9j(k - 1)] 

FJyjW 2 l i,y 2 {k) _ Fj (2- Fj )gj(k) 
~ Dy(fc-l) Dj{k-1)~ Dj(k-l) ' 

Noting < jlj < 2, we see that AV;(k) is guaranteed to be non-positive such that the 

boundedness of V;(k) is obvious, and immediately the boundedness of 9j(k) and bj\(k) is 
guaranteed. Taking summation on both sides of the above equation, we obtain 

oo y?(fc) 

L^-^^-^^iO) (5.26) 



which implies 



it lim o D-(fc-i) = °' or % {k) = *> {k) D i {k ~ l ) (5 ' 27) 
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with Uj (k) £ L 2 [0,oo) 
Define 



Yj(k) = [ Xj (k),Y (k)] 1 (5.28) 



where Yy(fc) is a vector holding states, at time k, of the 7th agent's neighbors. By (5.5) and (5.7), 
we have 

u j (k)=0[Y j (k + l)} 

tPjik) = 0[Yj(k)] (5.29) 

then it is obvious that 

Dj (k - 1) < 1+ \\cpjik - 1) || + \ Uj (k- 1)| 

= l + 0[Yj{k)}. (5.30) 

From (5.27) we obtain that 

yj(k) = o[l] +o[Yj(k)], j = l,2,...,N (5.31) 

Using o[X(k)] ~ o[xj(fc)] + o[x2(k)] + . . . + o[x„(fc)], we may rewrite the above equation as 

Y(fc)~diag(o[l],...,o[l])(G A + /)X(fc) 

+ [o[l],...,o[l]] T (5.32) 

where I is the n x n identity matrix. Substituting the above equation into equation (5.19), we 
obtain 

X(* + l) = (AG A + diag(o[l],...,o[l])(G A + 7))X(/c) 
+ [x*(k + l) + o[l],o[i\,...,o[l]] T . 

Since 

(AG A + diag(o[l], . ..,o[l])(G A + I))Y(k) -> AG A (5.33) 

as fc — > 00, noting p(AG A ) < 1, according to Lemma 5.1 and 

[x*(fc + l)+o[l],o[l],...,o[l]] r = 0[l] (5.34) 

from Lemma 5.3, we have 

X(fc + l) = 0[l]. (5.35) 

Then, together with equation (5.32), we have Y(k) = [o[l],. . .,o[l]] r , which implies 

\jj{k) -^0ask^co,j = l,2,...,N (5.36) 

which leads to X\ (k) — x* (k) — ► 0. 

Step 2: Next, we define a vector of the errors between each agent's output and the hidden 

leader's output as follows 

E(k) = X(k) - [1, 1,..., l] T Xl (k) = [e n (k),e 21 (k),...,e nl (k)] T (5.37) 
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where e-,\ (k) satisfies 

eil(fc+l) = x 1 (k + l)-x 1 (k+l) = 0, 

e jX (k + 1) = Xj(k + l)-xi(k + l) = Xj{k + 1) - x x {k + 1) + Xj(k + 1), 

j = 2,3,...,N. (5.38) 

Noting that except the first row, the summations of the other rows in the sub-stochastic matrix 
AG A are 1, we have 

[ai,...,i] T = AG A [0,1,...,1] T 

such that equations in (5.38) can be written as 



E{k + 1) = AGX{k) - AG A [0,l,---A] T x 1 {k + l) 
+ diag(0,l / ...,l)f(fc). 



(5.39) 



According to Assumption 5.1, we obtain 



E(k+1) = AG A (X{k) - [0,1, . . . ,l} T Xl (k)) 
+ [0,l,...,l] T (x 1 (/c)-x 1 (/c + l)) 

+[o[i],..., [ir 

= AGE(k) + [o[l],...,o[l]] T . 



(5.40) 



Assume that p' is the spectral radius of AG A , then there exists a matrix norm, which is denoted 
as || ■ ||p, such that 

\\E(k+l)\\ p <p'\\E(k)\\ p + o[l} (5.41) 

where p < 1. Then, it is straightforward to show that 

\\E(k + l)\\ p ->0 (5.42) 

as k — > oo. This completes the proof. ■ 

6. Summary 

The decentralized adaptive control problems have wide backgrounds and applications in 
practice. Such problems are very challenging because various uncertainties, including 
coupling uncertainties, parametric plant uncertainties, nonparametric modeling errors, 
random noise, communication limits, time delay, and so on, may exist in multi-agent systems. 
Especially, the decentralized adaptive control problems for the discrete-time multi-agent 
systems may involve more technical difficulties due to the nature of discrete-time systems 
and lack of mathematical tools for analyzing stability of discrete-time nonlinear systems. 
In this chapter, within a unified framework of multi-agent decentralized adaptive control, 
for a typical general model with coupling uncertainties and other uncertainties, we 
have investigated several decentralized adaptive control problems, designed efficient local 
adaptive controllers according to local goals of agents, and mathematically established the 
global properties (synchronization, stability and optimality) of the whole system, which in 
turn reveal the fundamental relationship between local agents and the global system. 
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1. Introduction 

New design tools and systematic design procedures has been developed in the past decade to 
adaptive control for a set of general classes of nonlinear systems with uncertainties (Krstic et 
al., 1995; Fradkov et al., 1999). In the absence of modeling uncertainties, adaptive controllers 
can achieve in general global boundness, asymptotic tracking, passivity of the adaptation loop 
and systematic improvement of transient performance. Also, other sources of uncertainty 
like intrinsic disturbances acting on measures and exogenous perturbations are taking into 
account in many approaches in order for the controllers to be more robust. 
The development of adaptive guidance systems for unmanned vehicles is recently starting to 
gain in interest in different application fields like autonomous vehicles in aerial, terrestrial as 
well as in subaquatic environments (Antonelli, 2007; Sun & Cheah, 2003; Kahveci et al., 2008; 
Bagnell et al., 2010). These complex dynamics involve a high degree of uncertainty (specially 
in the case of underwater vehicles), namely located in the inertia, added mass, Coriolis and 
centripetal forces, buoyancy and linear and nonlinear damping. 

When applying digital technology, both in computing and communication, the 
implementation of controllers in digital form is unavoidable. This fact is strengthened 
by many applications where the sensorial components work inherently digitally at regular 
periods of time. However, usual applications in path tracking of unmanned vehicles are 
characterized by analog control approaches (Fossen, 1994; Inzartev, 2009). 
The translation of existing analog-controller design approaches to the discrete-time domain 
is commonly done by a simple digitalization of the controlling action, and in the case of 
adaptive controllers, of the adaptive laws too (Cunha et al., 1995; Smallwood & Whitcomb, 
2003). This way generally provides a good control system behavior. However the role 
played by the sampling time in the stability and performance must be cautiously investigated. 
Additionally, noisy measures and digitalization errors may not only affect the stability 
properties significantly but also increase the complexity of the analysis even for the simplest 
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approaches based on Euler or Tustin discretization methods (Jordan & Bustamante, 2009a; 
Jordan & Bustamante, 2009b; Jordan et al, 2010). 

On the other side, controller designs being carried out directly in the discrete-time domain 
seem to be a more promising alternative than the translation approaches. This is sustained on 
the fact that model errors as well as perturbations are included in the design approach directly 
to ensure stability and performance specifications. 

This work is concerned about a novel design of discrete-time adaptive controllers for 
path tracking of unmanned underwater vehicles subject to perturbations and measure 
disturbances. The presented approach is completely developed in the discrete time domain. 
Formal proofs are presented for stability and performance. Finally, a case study related to a 
complex guidance system in 6 degrees of freedom (DOF's) is worked through to illustrate the 
features of the proposed approach. 

2. Notation 

Throughout the chapter, vectors are denoted in lower case and bold letters, scalars in lower 
case letters, matrices in capital letters. A function dependence on a variable is denoted 
with brackets as for instance F[x\. Also brackets are employed to enclose the elements of a 
vector. Elements of a set are described as enclosed in braces. Parentheses are only used to 
separate factors with terms of an expression. Subscripts are applied to reference elements in 
sequences, in matrices or sample-time points. In time sequences, one will distinguish between 
a prediction x n+ \ at time t„ from a sample x[t n ] = Xt n at sample time t n . Often we apply the 
notation for a derivative of a scalar function with respect to a quadratic matrix, meaning a 
new matrix with elements describing the derivative of the scalar function with respect to each 
element of the original matrix. For instance: let the functional Q be depending on the elements 

T j T 

Xj; of the matrix X in the form Q = (MXv 1 ) v 2 , then it has a derivative dQ/dX = M v 2 v 2 . 
Finally we will also make reference to dQ/dx meaning a gradient vector of the functional Q 
with respect to the vector x;, being this the column j of X. 

3. Vehicle dynamics 

3.1 Physical dynamics from ODEs 

Many systems are described as the conjugation of two ODEs in generalized variables, namely 
one for the kinematics and the other one for the inertia (see Fig. 1). The block structure 
embraces a wide range of vehicle systems like mobile robots, unmanned aerial vehicles (UAV), 
spacecraft and satellite systems, autonomous underwater vehicles (AUV) and remotely 
operated vehicles (ROV), though with slight distinctive modifications in the structure among 

them. 

r 

Let t]= [x, y, z, q>, 8, tp] be the generalized position vector referred on a earth-fixed coordinate 
system termed O', with displacements x, y, z, and rotation angles <p, 9, xp about these directions, 
respectively. The motions associated to the elements of rj are referred to as surge, sway, heave, 
roll, pitch and yaw, respectively. 

T 

Additionally let v= [u, v, w, ip, q, r] be the generalized rate vector referred on a vehicle-fixed 
coordinate system termed O, oriented according to their main axes with translation rates 
u, v, w and angular rates p, q, r about these directions, respectively. 
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The vehicle dynamics is described as (see Jordan & Bustamante, 2009c; cf . Fossen, 1994) 

v=A/f 1 (-C[v]v-D[|v|]v-gfo] + t c +t) (1) 

*=/[?] (v+vc). (2) 

Here M, C and D are the inertia, the Coriolis-centripetal and the drag matrices, respectively 

and / is the matrix expressing the transformation from the inertial frame to the vehicle-fixed 

frame. Moreover, g is the restoration force due to buoyancy and weight, t is the generalized 

propulsion force (also the future control action of a controller), r c is a generalized perturbation 

force (for instance due to wind as in case of UAVs, fluid flow in AUVs, or cable tugs in ROVs) 

and v c is a velocity perturbation (for instance the fluid current in ROVs /AUVs or wind rate 

in UAVs), all of them applied to O. 

Also disturbances acting on the measures are indicated as Si] and Sv, while noisy measures 

are referred to as tjg and vg, respectively. 

Particularly, in fluid environment the mass is broken down into 

M = M b + M a , (3) 

with Mf, body mass matrix, M„ the additive mass matrix related to the dragged fluid mass in 
the surroundings of the moving vehicle. 

For future developments in the controller design, it is convenient to factorize the system 
matrices into constant and variable arrays as (Jordan & Bustamante, 2009c) 

6 

C[v] = £Q.xC ti [y] (4) 

i=\ 

D[|v|]=D ; + £ D ?>«l (5) 

gfo] = B i si[y]+ B 2g2[ti], (6) 

with ". x " being an element-by-element array product. The matrices Q, D;, D„ jr B\ and B^ are 
constant and supposed unknown, while C Vi ,gi and g2 are state-dependent and computable 
arrays and P; is an element of v. 

The generalized propulsion force T applied on O is broken down into force components 
provided by each thruster. These components termed /, are arranged in the vector f which 
obeys the relation 

t=B T (BB T ) ~ t, (7) 

with B a commonly rectangular matrix that expresses the transformation of T into these thrust 

components. 

On the other hand, f is related to a strong nonlinear characteristic which is proper of each 

thruster. Specially for underwater vehicles this is modelled by (cf. Fossen, 1994) 

f=K 1 (|n|.n)-K 2 (|n|.v.) / (8) 

where K\ and K2 are constant matrices accounting for the influence of the thruster angular 
velocity n and the state v a related to every thruster force component in f . 
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The thruster dynamics usually corresponds to a controlled system with input n re ( and output 
n given generally by a linear dynamics indicated generically as some linear vector funcion k 
in Laplace variable form 

n=k[n,. e/ ,v], (9) 

where n re c is the reference angular velocity referred to as the real input of the vehicle 

dynamics. 

Usually in the literature it is assumed that the rapid thruster dynamics is parasitic in 

comparison with the dominant vehicle dynamics. In the same way we will neglect this 

parasitics and so the equality n=iu will be employed throughout the chapter. 

Moreover, we will concentrate henceforth on disturbed measures t] s and v,5, and not on 

exogenous perturbations T c and v c , so we have set t c =v c =0 throughout the paper. Similarly 

v=v and t]=t] (see Fig. 1). For details of the influence of r c and v c on adaptive guidance 
systems see (Jordan and Bustamante, 2008; Jordan and Bustamante 2007), respectively. 

3.2 Sampled-data behavior 

For the continuous-time dynamics there exists an associated exact sampled-data dynamics 

described by the set of sequences {»/[£;], v[f,]} = < t] t .,vt t > for the states tj[t] and v[t] at sample 
times tj with a sampling rate h. 

On the other side, we let the sampled measures for the kinematics and positioning state 
vectors be disturbed. So this is characterized in discrete time through the noisy measurements 
in the sequence set {%■[£;], v^f;]} = < t] s ,Vg t > as illustrated in Fig. 1. 



I ° \ov 




Fig. 1. Adaptive digital control system for underwater vehicles (UV) with noisy measures, 
model errors and exogenous perturbations 

3.3 Sampled-data model 

Usually, sampled-data behavior can be modelled by n-steps-ahead predictors (Jordan & 
Bustamante, 2009a). Accordingly, we attempt now to translate the continuous time dynamics 
of the system into a discrete-time model. The ODEs in (l)-(2) can be described in a compact 
form by 



v = M p[t],v]+M t 



(10) 

(ii) 
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with p and q being Lipschitz vector functions located at the right-hand memberships of (1) 
and (2), respectively. Here no exogenous perturbation was considered as agreed above. 
Let us contemplate an approximation of first order of an Adams-Bashforth approximator 
(Jordan & Bustamante, 2009b). It is valid 

v„+j = v^+ZiiVr 1 (p^+T„) (12) 

Vn+1 = Vt n + h< lt n , (13) 

where t] n+ i and v n+ j are one-step-ahead predictions at the present time t„. Moreover, r n is the 
discrete-time control action at t n , which is equal to the sample T[t n ] because of the employed 
zero-order sample holder. 
More precisely it is valid with (l)-(2) 

6 

Vt„ = - E C i- x c */, n,-Div t „- (14) 

i=\ 
6 
- E D <?< 1% ln,- B l gl,„ - B 2 g2,„ 

qt„ = h,yt n (15) 

where C Vj means C B/ [vfj, gj, and g2, mean gi[t] t ] and %i\r\ t } respectively, } t means 

/ [t] t ] and V{ is an element of v^ . Similar expressions can be obtained for the other sampled 
functions pt t and q^ in (18)-(19). Besides, the control action t is retained one sampling period 
/z by a sample holder, so it is valid r n = tt„ ■ 
The accuracy of one-step-ahead predictions is defined by the local model errors as 

£ V n +l = V t n+1 -V n+1 (16) 

E 'h, + i = 1t n+1 -*ln+l> (17) 

with e„ ^ , £ Vll+1 G O [h] and O being the order function that expresses the order of magnitude 
of the sampled-data model errors. It is noticing that local errors are by definition completely 
lacking of the influence from sampled-data disturbances. 

Since p and q are Lipschitz continuous in the attraction domains in v and r\, then the samples, 
predictions and local errors all yield bounded. So it is valid the property v„ + j— >Vf ii+I and 

In+r+Vtnv for/2^0. 

Next, the disturbed dynamics subject to sampled-data noisy measures is dealt with in the 

following. 

3.4 1st-order predictor with disturbances 

The one-step-ahead predictions with disturbances result from (18) and (19) as 

v„+2 = v^+Sv^+hM- 1 {ps,,+r„) (18) 

Vn+l = It+^t+Hs,,,, (19) 
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where Vf n +<5vt n =vg t and r\ t +Si] t =t]g are samples of the measure disturbances (see Fig. 
1), and pg t and q^ ( are perturbed functions defined as p$ t =P 



vt n +5v t „,rit +3>lt 



and 



%„ = i 



v t„+ Sv t„,ti t + St it 



3.5 Disturbed local error 

Assuming bounded noise vectors <5v; and Srjp we can expand (18) and (19) in series of Taylor 
about the values of undisturbed measures v[f„] and t][t n ]. So it is accomplished 



I d ?s *„. , 3 P* 



+ ^f<5v t „ + ^^ tn +o[Jv']+o[^l) (20) 

£ >7„ +1 = ^^^U^ f ^f ['»]<*'» + 

+ ^[f»]H„+o[^ 2 ] + o[^ 2 ]j, (21) 

where e?, ,, and e„ are the model local errors and ASvt ,.=6vt^—&vt and ASn t = 
Srj t —Sq t ■ The functions o are truncating error vectors of the Taylor series expansions, all of 

them belonging to 0[h]. Moreover, -£*-, -£*-, -3*- and -3^ are Jacobian matrices of the system 
which act as variable gains that strengthen the sampled-data disturbances along the path. 

T T 

It is worth noticing that the Jacobian matrices --^ and -gf- in (20) will be obtained from the 
feedback law r n [t] t > v t„] OI me adaptive control loop. 

4. Sampled-data adaptive controller 

The next step is devoted to the stability and performance study of a general class of adaptive 

control systems whose state feedback law is constructed from noisy measures and model 

errors. 

A design of a general completely adaptive digital controller based on speed-gradient control 

laws is presented in (Jordan & Bustamante, 2011). To this end let us suppose the control 

goal lies on the path tracking of both geometric and kinematic reference as r\ r and v r , , 

respectively. 

4.1 Control action 

Accordingly to the digital model translation, we try out the following definitions for the exact 
path errors 

Vt„ = iu+ dt ik-% n (22 ) 

v f „ = v f „ +^-7^ i] riu +J s l K p q tn . (23) 
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T -1 

where K„ = K„ > is a design gain matrix affecting the geometric path error and J, means 

/ lit +<^/t ]. Clearly, if j/ t =0, thenby (23) and (2), it yields vt„+5vt„— v ri =0. 
Then, replacing (18) and (19) in (22) for t n+ \ one gets 



***» = V ~ h h«K K P) lu+lr,, -% n+1 + S 1t„ +1 - S 1t„ ( 24 ) 

+ e 'i„ + ,+ h {J*.**, + hM„+hhl%„) ■ 

Similarly, with (18) and (19) in (23) for t n+ i one obtains 

vt„ +1 =vtn+kl% n -kl +1 Vr tn+1 -kl K pnt n + (25) 

+Jsl +1 K pyt„ +1 + £ v» + i+t v t„ + i-£vt„+hM~ 1 {ps t „+r„) . 

We now define a cost functional of the path error energy as 

Qtn = f 1t„ t lt„+vt,vt„, (26) 

which is a positive definite and radially unbounded function in the error vector space. Then 
we state 

AQ f „ = Q Wl - Q tn = (27) 

1 - h hJsl K p) vt+ h (h,v t „ + hjv^+hJslx, 



+ (ju, +lsl %„ -Js! u+1 K + i~ J i K v\ + h[ +1 K P*lt n+1 
+ hM ' {p St „ +r n ) + Sv t „ + , - Sv tn +e Vn+ , ) -v fn . 



The ideal path tracking demands that 

lim AQ f „ = lim (Q tn+1 - Q t „) = 0. (28) 

Bearing in mind the presence of disturbances and model uncertainties, the practical goal 

would be at least achieved that { AQf n } remains bounded for t n —¥ oo. 

In (Jordan & Bustamante, 2011) a flexible design of a completely adaptive digital controller 

was proposed. Therein all unknown system matrices (C,, D„., D;, B\ and B2) that influence the 

stability of the control loop are adapted in the feedback control law with the unique exception 

of the inertia matrix M from which only a lower bound M is demanded. In that work a 

guideline to obtained an adequate value of that bound is indicated. 

Here we will transcribe those results and continue afterwards the analysis to the aimed goal. 

First we can conveniently split the control thrust r n into two terms as 

T„ = Ti„+T 2 „, (29) 
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T 2 „ = -K v v tn -^M[J s ; n ii rtn +Ji iti K p ti tn - 



where the first one is 



T 

with K v = K v > being another design matrix like Kp, but affecting the kinematic errors 
instead. The vector tx, is 

6 
r ^ ( „ = E U i- X Cp . t V -5*„ + U 7 v S t „ + (31) 

! = 1 

6 
+ E ^K, K tn +Ui4gl 4f +Lfi 5 g2 , 
1=1 

where the matrices Ui in r^ ( will account for every unknown system matrix in px in order 

to build up the partial control action Tj n . Moreover, the !i,'s represent the matrices of the 

adaptive sampled-data controller which will be designed later. Besides, it is noticing that tx t 

and p<s ( contain noisy measures. 

The definition of the second component T2 n of T„ is more cumbersome than the first 

component Tj n . 

Basically we attempt to modify AQf n farther to confer the quadratic form particular properties 

of sign definiteness. To this end let us first put (30) into (27). Thus 

&Qt n = Qt n+ , - Qt,= (32) 



+%„ -% n+1 + £ n„ +1 +H„ +1 -H„ j - n t „ + 

+ (\ +fsl % n -fs] n+l % n+1 -hi K pv tn +fs] n+1 k pH +1 ~ 

2 

+hM~' {p s -t s ) +hM~\ 2n +fat n+1 -5v tn +e Vn+1 ) -v tii , 



T -l -r-1 



where the old definition of /^ = / [t] t -\-Si] t ] can be rewritten as 

Jsl =hl+ A/,; 1 . (33) 

Now defining an motion vector function (combination of acceleration and velocity) in the 
form 

s t„ =Jsl % n - fs[ +l %„ + i ~ } K K v^t n + Js! :i+1 K pVt„ +1 , (34) 
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the (32) turns into 

AQ t „ = Qt n+1 - Qt n = (35) 

(I - hK p ) v fu - hJtM^Kpq^+h (/ t „v f „ + n n ) +kM„+hAJt!% n 



+ ((l- hM^Kv) v tn + (i - M^M] s t - 



+h(p Sh -M~\s hl ) +hhf 1 r 2 „ +6vt n+1 -6vt,,+e Vll+1 j -v tii . 
From this expression one achieves 

AQ tn = a{Mr 1 T 2 „) 2 W Nf 1 r ln +c+ (36) 

+tj tn VaKp (VaK p - 21) tj tn + 

+vJy^*(v / flJ s C*-27)v f „ + 

+/AQi„ K, +i ' £ v„ + i >%„+, '<K +5 ], 

where K v is an auxiliary matrix equal to K v = M K v . The polynomial coefficients a, b and c 
are 

a = h 2 (37) 

b = 2h{I-hK v )v til +2h (i-M^M) s tli +2hM~ 1 (p Sli -r s J 



+ 2h Uv tn+1 - Sv t „+e v „ +1 I (38) 

2 

c = ^ 2 (/f„v f „+^ ( J + (39) 

T 

+ K.-OK.-0 + 

T 

+ 2 ((I- WC p ) Si? fn ) (/*(/ f „v f „+^ lR )+^,„ -V r ,„ +] ) + 

+ (j-A^'m) Sf n +/i 2 M _1 (p^-r^) 2 + Uv f „ +J -Sv fll +e v „ +1 

+ 2 ((/- M _1 m) Sf „+/iM _1 (p^-r^)) Uv ln+7 - Sv tn +e Vn+1 j 
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and /aQi i s a sign-undefined energy function of the model errors and measure disturbances 
defined as 

/AQi„ t e f„ +! > £ v„ + i Mn+i ' Sv t„ + i ) = (40) 

e i„ + i +St lt„+i - St lt„ - h Jt„ A h!, K pf]t„ + h,5vt„ +h,Ah!,il ri:i 



irt. 



X 



+2 1(1- MC p ) r, t +h \J tn w tn + 9 n J + i, r - % 

(z'l^+fyt^-^-hhAhtKptlt,, + hM„+hAJKV ri „) ■ 

Clearly, there are many variables involved like the system matrices, model errors and measure 
disturbances which are not known beforehand. 

The idea now is to construct T2 n so that the sum a(M^ 1 x% ) +b M T% +c in (36) be null. As 
there are many variables in the sum which are unknown, we can construct an approximation 
of it with measurable variables. So, it results 

a(M~\ 2 }) +b, 1 M~ 1 T2„+c„=0. (41) 

Now, the polynomial coefficients a b B and c„ are explained below. Here, there appear three 
error functions, namely f&n l , and the new functions /aq 2 an d fu, > au containing noisy and 
unknown variables which are described in the sequel. 
The polynomial coefficients result 



a = a=h (42) 



b„ = 2h{I-hK v )v tn + 2hM-\f St -t K ) (43) 



c>i = h M/<„v f „+^,J + (A/^v^J +2[} tn v t „+ij nn J Nstvt n )+ (44) 

T T 

+2h (/ fi ,v,„+^.J (? r „-?r w J + 2h { A ht,v t „) (%„-%„ +1 

T 

T 

+2 ((7 - hK p ) n t: ) (h (kn,+fl riii ) +%„-V %+} )+ 2 (( I - hK p) \) [Wuju] 
+KM- 1 {v K -h tn ) 2 +2(hM-\p Stn -t K f) (I-hK*)v t „, 
with p^ being an estimation of p^ in (14) given by 

Ps tn =M Vt ^P^-r n . (45) 
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The second component T2 of T„ was contained in the condition (41) like a root pair that 
enables AQt„ be the expression (47). It is 



/ -b 1 bb-4ac \ 

^= M [^ ± 2ni^^ 1 )' (46) 

with 1 being a vector with ones. 

With the choice of (41) and (46) in AQt„ one gets finally 

AQt, =n ln hK v (hK v -2l)n tn + (47) 

T 

+v t HK* v (hK* v - 21) v f „ +f& Qln [e Vn+1 ,e Vn+1 ,Sijt n+1 M nU ] + 

+/AQ2,, [ £ v„ +1 ' e ^ J*lt n+ , M n+1 } +fu,„ [{Uf- Ui ),M- 1 M}. 

The matrices Lf* that appear in fjj. take particular constant values of the adaptive controller 
matrices LTj's. They take the values equal to the system matrices in (l)-(2) (Jordan and 
Bustamante, 2008), namely 

U\ = Q,withi = l,...,6 (48) 

U 7 = D, (49) 

U] = D ?j/ withi = 8,...,13 (50) 

U u = Bi (51) 

U l5 = B 2 . (52) 

Moreover, the error functions /aq 2 anc ^ fu, m (47) are respectively 



/AQ Z „ [ e V n+ i'^« + i' St lt„ + u^tn + i] = 2h (^ v f„+i -^t„+e v „ +1 ) x 



(53) 



^M-^M^A-*^ 



-2h 2 (A/ ft „V( n ) (% n -1r, ii+ ) -2fe 2 (h,vt„+%„) A/«„v f „ - 

T x V 2 

-2/z ((I - MC p ) tJt^AJst^ + f <Jv Wj - <5v t „+£-c„ +1 J + 
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with Ab= b b from (38) and (43), and 

f Uin [(Uf-U i ),M~ 1 M]= (54) 

V'Mbf (M-'Mb) 1 T (M- I M) T (M- 1 M)1 (V b) F T M -> MF 



4« 24a 2a 



) — y - b 4flC i+ 

+2(/i 2 (M- 1 -M- 1 )J< l ,v t , i +/! 2 M- 1 (p^-r t J-/! 2 M- 1 (p t , i -r^ 



lac 1 | 



1 M M ["2« ± 2«V 6 1 l+ hM (P^,-\,) + 



+ (i-AT'm) s]+2h (M' 1 (p K -t th )) (i-M^M) s tn + 
+2[h(M' 1 (p Sh -r li )) +Sj n (l-M" 1 M) jCl-hM-^Jvt,- 



-»M ^„-r,J -2(^(M ^ ( „-r,JJ J(J-^M K„)v t „. 

It is seeing from (40), (53) and (54), that the error functions go to lower bounds when Uj = U* 
(it is, when p^ =r s ), M M = I and Sr]t n+1 =6vt„ +1 =0. These bounds will ultimately depend 
on the model errors e^ and e Vll+1 only. 

It is noticing from (46) that the roots may be either real or complex. Clearly when the roots 
are real, (41) is accomplished. If eventually complex roots appear, one can chose only the real 

part of the resulting complex roots, namely T2„ = M. = £f-. The implications of that choice will 
be analyzed later in the section dedicated to the stability study. 

Finally, the control action to be applied to the vehicle system is r n = Tj + T2 with the two 
components given in (30) and (46), respectively. 

4.2 Adaptive laws 

According to a speed-gradient law (Fradkov et al., 1999), the adaptation of the system 
behavior occurs by the permanent actualization of the controller matrices IT,. 
Let the following adaptive law be valid for i = 1, ..., 15 

IT A IT T 9A Q f " /-cc-i 

U i„+i = "i„ _ ! i qjj. ' I 55 ) 

with a gain matrix T, = T- > and -.n.'" being a gradient matrix for Uj n . 
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First we can define an expression for the gradient matrix upon AQj u in (47) but considering 
that M is known. This expression is referred to the ideal gradient matrix 

1- A "'( M "")(w)'- 

T 

2 -t -l /drx 

-2hM M (p,„,-rO(4 

-2hM (I-hK v )vtJ^ 

Now, in order to be able to implement adaptive laws like (55) we have to replace the unknown 
M in (56) by its lower bound M. In this way, we can generate implementable gradient matrices 

which will be denote by -.„'" and is 

*§§• --»■** (M-r,) (||)'-_ 



-2hM M (p,„-r, n 



with the property 



where 



-2hM (I - hK v )v tn , 



3AQ,„ dAQt, 



dUi dUj 



(58) 



Alt,,, =^m-^;„ + <5 m -i b >> (59) 

and S M - 2 = U/f 7 ^ 1 - M~ T M~ l \ > and d M -i = (m _1 - M~'\ > 0. Here A, n and B, n are 

sampled state functions obtained from (56) after extracting of the common factors 5 M -i anc ^ 

S M -i , respectively. 

It is worth noticing that AQt n and AQ t , satisfy convexity properties in the space of elements 

of the U/s. 

Moreover, with (58) in mind we can conclude for any pair of values of If,-, say LL of U i , it is 

valid 

AQ ( „(U,)-AQ fn (li,. ) < g^ ' ; (Ui-U, ) < (60) 



dAQ tn (U { 
dU, 



< 7;; " (Ui-Ui). (6i) 



This feature will be useful in the next analysis. 
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In summary, the practical laws which conform the digital adaptive controller are 

^, +1 = ^-r,^. (62) 

Finally, it is seen from (57) that also here the noisy measures tfg and \g will propagate into 
the adaptive laws ->,,'" . 

5. Stability analysis 

In this section we prove stability, boundness of all control variables and convergence of the 
tracking errors in the case of path following for the case of 6 DOF's involving references 
trajectories for position and kinematics. 

5.1 Preliminaries 

Let first the controller matrices IT, 's to take the values U i 's in (48)-(52). So, using these constant 

system matrices in (l),(4)-(6) and (14), a fixed controller can be designed. 

For this particular controller we consider the resulting AQ f from (47) accomplishing 

AQl = y t HK v {hK p - 27) \ + (63) 

+v tn hKl(hKl-2l)v tn + 

+fAQ n [ £ 1n»' £ V n+1 ,Stltn^t n ,M~ 1 M], 

where / A * is the sum of all errors obtained from (47) with (53) and (54). It fulfills with 

Pit, =r 4, 

/a*Q„ = fAQu + f&Qi n IPS,,, = r <5,„ ] +/u„, [Ps ln =*5 ln ] ■ (64) 

Later, a norm of / a *q will be indicated. 

Since ( £^ ; +6r/ t —Sr/ t ) , f Svt n+1 — Sv tn +e Vll+1 G lco and M M € loo, then one concludes 

/ A *Q n € loo as well. 

So, it is noticing that AQ t < 0, at least in an attraction domain equal to 



B 



with Bq a residual set around zero 



{^v^K'nSj}, (65) 



Bo= {vt B >vt n eK 6 /AQ' tn - f A * Qn < O} (66) 



and with the design matrices satisfying the conditions 

2 
h 



I > K p > (67) 



\l > ¥1 > 0, (68) 

h 
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which is equivalent to 

Im>Im> K v > 0. (69) 

h n 

The residual set B depends not only on e^ and s Vn+1 and the measure noises Sfjt n and 5vt n , 

but also on M M. In consequence, B Q becomes the null point at the limit when h — ¥ 0, Si]t„ , 

Sv tn -> and M = M. 

5.2 Stability proof 

The problem of stability of the adaptive control system is addressed in the sequel. Let a 
Lyapunov function be 



1 



15 6 



1 



Q,+iEE(^ ir 1 w w - (70) 

i=i 1=1 ^ ' i«+i " +i 

— i 



15 



1=1 ;=1 



with ( u, ) = f Uj— u. I , where u; and u ; are vectors corresponding to the column j of the 

adaptive controller matrix Uj and its corresponding one Lf ; in the fixed controller, respectively. 
Then the differences AVj ;i = Vj u+1 — Vj n can be bounded as follows 



1 



15 6 



AVi. = AQ f „ + - £ E (Au[ j r: ( (« ; - j ; + (u ; ) ; ) (71) 

1=1 j=l 



l„+l 

15 6 . _. , . . i 15 6 



AQ, + E E W). r: 1 (S,) -\ZZ K) ( r:' (a U/ 



i„ 



**- gt(^)'ft) 

< AQ^ <0inSnS2, 

with ( Au, J a column vector of ( Uj ii+1 — Ui n ) . 

The column vector ( Au; ) at the first inequality was replaced by the column vector 



— Tj I g~'" I and then by — T, I g '" J in the right member according to (58) and (60)-(61). 

So in the second and third inequality, the convexity property of AQt„ in (60) was applied for 

any pair (W = U in ,U" = !!,*). 

This analysis has proved convergence of the error paths when real square roots exist from 

b„b„-4flc„ of (46). 
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T 

If on the contrary Mc n > b„b n occurs at some time t n , one chooses the real part of the complex 
roots in (46). So a suboptimal control action is employed instead, In this case, it is valid 

T2,=^M"b„ = ==-(/-«C*)v,„. (72) 

So it yields a new functional AQ" in 

AV t „ < AQl = AQl + c„ - ^Kbn <OinBnBo, (73) 

where AQ t is (63) with a real root of (46) and B is a new residual set. It is worth noticing 
that the positive quantity I c n — j|jb n b„ I can be reduced by choosing h small. Nevertheless, 
B results larger than B in (71), since its dimension depends not only on e,j and £v„ +1 but 
also on the magnitude of ( c„ — pb„b B I . 
This closes the stability and convergence proof. 

5.3 Variable boundness 

With respect to the boundness of the adaptive matrices 17; 's it is seen from (57) that the 
gradients are bounded. Also the third term is more dominant than the remainder ones for h 
small (/( < < 1), and so, the kinematic error Vf n influences the intensity and sign of 3AQ t /3L7, 
more significantly than the others. From (62) one concludes than the increasing of |L7,| may 
not be avoided long term, however some robust modification techniques like a projection zone 
can be employed to achieve boundness. This is not developed here. The author can consult 
for instance (Ioannou and Sun, 1995). 

5.4 Incidence of model errors and noisy measures 

It is seen in (66) that the residual set B Q is conformed by the perturbation error function / A * . 
In this section some guidelines can be given for a proper selection of the design parameters in 
order to diminish the incidence of model errors and noisy measures. This concerns the design 
matrices K„ and K v as well as operation parameters like the cruise vehicle velocity and the 
control action self. 
To this end, let the sign-undefined term / a *q be upper bounded by 

|/aq„ I < If, 1 K +1 +H„ +! -H„ I + 1*, I \h, I \&hl K v\ + A h!,%„ I + 1*, I \h, I \tot, I + (74) 
+ |f,| \AJst„v t ,\ + |f 3 | K +1 +<K +1 -<KJ + 

2 

lh | A/f ' t 2 „ | \£v„ +1 +6vt„ +1 - <5v f „ | +h 2 \Aht n vt„ \ 

+ | £ '/„ +1 + Sl 1t„ +1 - s Vt„ - h h, A h!, K pVt„ +h Sv t„ +h„ A/;7 7,.,„ | 
+ \ e v n+1 +5-vt n+1 -5v ln \ + 

+ |/ 4 | Ii-m^mL 



2 
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where f , ,f , and f 3 are bounded vector functions 

T 

< = 2 ( (' - ^p) ?*. +* (ft. ^ + K ) + %-%, l+1 ) ( 75 ) 

< = -2" (h {% n -1 %+1 ) +h {h,vt n +V rin ) + (I-AK P ) v.,)' (76) 

< = ((l-M _1 M)s f „) , (77) 
and /4 is also a bounded scalar function which can be identified from (54) with p<s ( =tg as 

fu t [ps t „=T K ] = (78) 

M"Mb) T (M"M"b) l T (M- ! M) T (M- ! M)l(b T b) ^V'Mb 



45 245 25 



1 ,/b T b-45c 



Tb ((m-'m) (m-'m) -M-'M) i. ^ " 6 1+ 

T 

+2 ^(M^-M -1 )^^-/! (/-A'T'm) s f „) M~ 't 2 „ + 

2 T 

+ (i-M _1 m) s^+2 s£ (I-ATm) {I-hM' 1 K v )v t „, 



where the norm 7 — M M is implicitly contained herein. 

Clearly, if h — > 0, (5i/f„, <Jvf n — ¥ and M = M, then / a *q tends to zero conjointly. 

At the first glance in (64) and (53), one notices that the inertia matrix M appears in f 3 . So, it is 

valid 

j^< (l-A min [M _1 M])|| (79) 

which does reveal that the influence of M on f 3 is reduced whatever M be a good estimate of 
M, it is when A,^^ \M M be close to one. Besides, no dependence of the sampling time h 
on i 3 is observed any longer in the tightest bound in (79). 

Since the body inertia matrix M;, is plausible to be good estimated, we can choose M = My. 
So it is valid 



If.. 



\-l 



<1- (M b + M fl )- |M 6 |. (80) 

l s f»l ' ' 

Particularly, AUVs are designed with hydrodynamically slender profiles, they have 

commonly much more smaller values of M a than in the case of ROVs. In this sense, it is 

expected that the uncertainty M a affect more the steady-state performance in ROVs than in 

AUVs. 

The same analysis can be carried out for fjin (74). In particular, a choice of Kp in i 1 that is as 

close as possible to the value I/h (see (67)), will reduce partially |£, |. Analogously, the same 

result for Kp could be obtained from f 2 . 
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On the other side, one sees that small differences of ( t] r —if \ or equivalently small values 

of i] r have the influence of decreasing | f , | as well. Since the quantity t] r s± t ( assumes 

small values for h small, then large cruise velocities do not affect the performance if the 
sampling time is chosen relatively small. 

Besides, the term hjt„vt n in f 2 leads to the same conclusion about the effect of h. However, 

it is interesting to stress the fact that appears in vehicle rotations which may rise the norm 

of Jt„ [<p, 0, ip] considerably when the pitch angle goes above about 30° (Jordan & Bustamante, 

2011). 

The scalar function f±, whose bound is implicitly included in (78) gets small when particularly 

the vector b is small (this means also r n , small), and the motion vector function Sf„ is also 

rather moderate. 

Finally, there is the term 2hr2„ in (74) that also contributes to increase /aq 2 particularly when 

saturation values of the thrusters are achieved. Since T2 is fixed by the controller, the only 

countermeasure to be applied lays in the fact that the controller always choose the lower T2 

of the two possible roots in (46). So, the perturbation energy /aq 2 i s reduced as far as possible 

by the controller. 

From (63) one can draw out that the choice K v = \ Mf, in the negative definite terms is much 

more appropriate to increase the negativeness of AQ t . Equally the choice of K„ in the same 

manner helps the trajectories to get the residual set more rapid. 

Besides, the model errors and noisy measures (£i>„ +1 +Svt n+1 — 5vt„ +1 ) and 

(e„ +<5>/t — ^ v t„ + i ) enter linearly and quadratically in the energy equation (74). As 
they are usually small, only the linear terms are magnified /attenuated by f } , f 2 , i 3 and T2 , 
while f/j. impacts nonlinearly in t„ 2 and st n as seen in (54). 

5.5 Instability for large sampling time 

Broadly speaking, the influence of the analyzed parameters will play a role in the instability 
when (on the chosen h is something large, even smaller than one, because the quadratic terms 
rise significantly to turn to be dominant in the error function / A * . 

The study of this phenomenon is rather complex but it generally involves the function AQ f 
in (63) and / A * Qn in (64). 
For instante, when 

f & * Q >y t JiK p (hK v - 27) \ +v]hK v (hK v - 21) v tn , (81) 

the path trajectories may not be bounded into a residual set because the domain for the 
initial conditions in this situation is partially repulsive. So, depending on the particular initial 
conditions and for h > > the adaptive control system may turn unstable. 
In conclusion, when comparing two digital controllers, the sensitivity of the stability to h is 
fundamental to draw out robust properties and finally to range them. 

6. Adaptive control algorithm 

The adaptive control algorithm can be summarized as follows. 

Preliminaries: 

1) Estimate a lower bound M , for instance M = Mj (Jordan & Bustamante, 2011), 
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2) Select a sampling time h as smaller as possible 

3) Choose design gain matrices Kp and K v according to (68)-(69), and simultaneously in order 
to reduce / a *q and AQ t (see related commentary in previous section), 

4) Define the adaptive gain matrices F, (usually T,- = oc t l with a,- > 0), 

5) Stipulate the desired sampled-data path references for the geometric and kinematic 
trajectories in 6 DOF's: tj r and v r , , respectively (see related commentary in previous section). 
Continuously at each sample point: 

6) Calculate the control thrust r n with components Tj^in (30) and T2„ (46) (or (72)), 
respectively, 

7) Calculate the adaptive controller matrices (56) with the lower bound M instead of M. 
Long-term tuning: 

7) Redefine Kp, K v and h in order to achieve optimal tracking performance. 

7. Case study 

7.1 Setup 

With the end of illustrating the features of our control system approach, we simulate a 
path-tracking problem in 6 DOF's for an underwater vehicle in a planar motion with some 
sporadic immersions to the floor. 

A continuous-time model of a fully-maneuverable underwater vehicle is employed for the 
numerical simulations. Details of this dynamics are given in (Jordan & Bustamante, 2009c). 
The propulsion system is composed by 8 thrusters, distributed in 4 vertical and 4 horizontal. 
The simulated reference path rj r and the navigation path rj are reproduced together by means 
of a visualization program (see a photogram in Fig. 2). The units for the path run away are in 
meters. 

Basically the vehicle turns around a planar path. At a certain coordinate A it leaves the plane 
and submerses to the point A' for picking up a sample (of weight 10 (Kgf)) on the sea floor 
and returns back to A with a typical maneuver (backward movement and rotation). Then it 
continues on the planar trajectory till the coordinate B in where it submerses again to the point 
B' in order to place an equipment on the floor (of weight 20 (Kgf)) before to retreat and turn 
back to B and to complete finally the cycle. The vehicle weight is about 60 (kgf). 

Additionally to the geometric path, the rate function v r (t) = J~ 1 (tl r )i] r {t) along it, is also 

specified, with short periods of rest at points A and B' before beginning and after ending the 

maneuvers on the bottom. 

At the start point of the mission (represented by O in Fig. 2), it is assumed for the adaptive 

control there is no information available about the vehicle dynamics matrices. Moreover, the 

maneuvers at stretches A- A 1 and B-B' imply considerable changes of moments acting on the 

vehicle in both a positive and negative quantities. 

The reference velocity is programmed to be constant equal to 0.25(m/s) for the advance and 

as well as for the descent/ascent along the path. This rate will be referred to as the cruise 

velocity. 

By the simulations, the adaptive control algorithm summarized in the previous section, is 

implemented. It is coupled with the ODE (l)-(2) for the vehicle dynamics, whose solution is 

numerically calculated in continuous time using Runge-Kutta approximators (the so-called 

ODE45). The computed control action is connected to a zero-order sample&hold previously 

to excite the vehicle. 
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Fig. 2. Path tracking with grab sampling at coordinate A', and with placing of an equipment 
on the seafloor at coordinate B 

7.2 Design parameters 

The most important a-priori information for the adaptive controller design is the 
ODE-structure in (l)-(2) but not its dynamics matrices, with the exception of the lower bound 
for the inertia matrix M. This takes the form 



M = M b + M a 



with the components: the body matrix M;, and the additive matrix M„ given by 



M b = M hn + 6{t - t A ,) M bA+ - 5{t - t B ,) M m - 
M a = M a „ + S(t - t A r) M„ A+ - S(t - t B >) M aA -, 



(82) 



(83) 
(84) 



where M/, and M a „ are nominal values of Mj, and M a at the start point O, and 
M;,A-/Mf,A+,M fl A+ and M„a- are positive and negative variations at instants t A i and tgi on 
the points A' and B' of Fig. 2. Here S(t — f,) represents the Dirac function. 
For our application M;, is determinable beforehand experimentally and it is set as the lower 
bound M for the control and adaptive laws. In the simulated scenario, Mj,a- is assumed 
known because it is about of an equipment deposited on the seafloor. In the case of M bA +, 
M„a+ an d even M aA - , we depart from unknown values. 

The property of M a > is not affected by the sign of M aA + and M aA - , which may be positive 
and negative as well. For that reason, a valid lower bound is chosen as M = M{, — Mj, A - . 
Taking into account the simulation setup for the weight changes (the weight picked up from 
the seafloor at t A i and the second weight deposited on the seafloor at fg/), the lower bound for 
Mis 

M = diag(60, 60, 60, 5, 10, 10), (85) 
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and the mass variations are 

M b&+ = diag(10, 10, 10, 0.6250, 4.2250, 3.6) (86) 

M bA - = diag(20, 20, 20, 1.25, 1.25,0) (87) 

M aA+ = diag(6.3, 15.4, 0.115, 0.115, 0.261, 0.276) (88) 

M flA - = diag(12.6,30.8,0.23,0.23,0.521,0.551). (89) 



The design gain matrices for the controller are 

(90) 



K p = diag(5,5,5,5,5,5) 



K v = diag (300,300,300,25,50,50) 

and the adaptive gain matrices about 

Ti = I. (91) 

Finally we have proposed a sampling time h = 0.2(s). 
All quantities are expressed in the SI Units. 

7.3 Control performance 

Here the acquired performance by the autonomously guided vehicle under the described 
simulated setup will be evaluated. First, in Fig. 3, the path error evolution corresponding 
to every mode with their respective rates is shown for the different transient phases, namely: 
the controller autotuning at the initial phase (to the left), the sampling phase on the sea bottom 
at A' (in the middle), and the release of an equipment on the floor at B' (to the right). 
The largest path errors had occurred during the initial phase because the amount of 
information for the control adaptation was null. Here, the longest transient took about 5(s) 
which is considered outstanding in comparison to the commonly slow open loop behavior. 
Later, after the mass changes, the path errors behaved much more moderate and were 
insignificant in magnitude (only a few centimeters or a few hundredths of a radian according 
to translation /rotation). Among them, the errors in the surge, sway and pitch modes (x, z and 
6) resulted more perturbed than the remainder ones because they were more excited from the 
main motion provided by the stipulated mission. In all evolutions the adaptations occurred 
quick and smoothly. 

The same scenario of control performance can be observed in Fig. 4 from the side of the 
velocity path errors for every mode of motion. Qualitatively, all kinematic path errors were 
attenuated rapid and smoothly in the autotuning phase as well as during the mass-change 
periods. The magnitude of these errors is also related to the rapid changes of the reference v re t 
in the programmed maneuvers. 

In the Fig. 5, the time evolution of the actuator thrust for two arbitrarily selected thrusters 
(one horizontal and one vertical) is shown. Analogously as previous results, the forces are 
compared within the three periods of transients. One observes that the intervention of the 
controller after a sudden change of mass occurred immediately. Also the transients of these 
interventions up to the practical steady state were relatively short. 

Fig. 6 illustrates the time evolution of some controller matrices !i,. To this end, we had chosen 
the induced norm of M% which is partially related to the adaptation of the linear damping. 
One sees that the norm of M% evolved with significative changes. In contrast to analog 
adaptive controllers of the speed-gradient class, here the U;'s do not tend asymptotically to 
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Fig. 3. Position path errors during transients in three different periods (from left to right 
column: autotuning, adaptation by weight pick up and adaptation by weight deposit) 

constant matrices because of the difference between M and M in (58)-(59) (cf. Jordan & 
Bustamante, 2009c). 

8. Conclusions 

In this paper a novel design of adaptive control systems was presented. This is based on 

speed-gradient techniques which are widespread in the form of continuous-time designs 

in the literature. Here, we had focused their counterparts namely sampled-data adaptive 

controllers. 

The work was framed into the path tracking control problem for the guidance of vehicles 

in many degrees of freedom. Particularly, the most complex dynamics of this class 
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Fig. 4. Velocity path errors during transients in three different periods (from left to right 
column: autotuning, adaptation by weight pick up and adaptation by weight deposit) 

corresponding to unmanned underwater vehicles was worked through in this work. Noisy 
measures as well as model uncertainties were considered by the design and analysis. 
Formal proofs for stability of the digital adaptive control system and convergence of the path 
error trajectories were presented and an extensive analysis of the control performance was 
given. 

It was shown that it is possible to stabilize the control loop adaptively in the six degrees of 
freedom without any a-priori knowledge of the vehicle system matrices with the exception of 
a lower bound for the inertia matrix. 

Providing the noisy measures remain bounded, the adaptive controller can reduce 
asymptotically the path errors up to a residual set in the space state. The residual set contains 
the null equilibrium point and its magnitude depends on the upper bounds of the measure 
noises and on the sampling time. This signalizes the quality of the control performance. 
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However, as generally occurs by digital controllers, it was observed that a large sampling 

time is an instabilizing factor. 

It was also indicated the plausibility of obtaining a lower bound of the inertia matrix by simply 

calculating the inertia matrix of the body only. 

We will emphasize that the design presented here was completely carried out in the discrete 

time domain. Other usual alternative design is the direct translation of a homologous but 

analog adaptive controller by digitalizing both the control and the adaptive laws. Recent 

results like in (Jordan & Bustamante, 2011) have shown that this alternative may lead to 

unstable behaviors if the sampling time is particularly not sufficiently small. This fact stands 

out the usefulness of our design here. 

Finally, a case study was presented for an underwater vehicle in simulated sampling mission. 

The features of the implemented adaptive control system were highlighted by an all-round 

very good quality in the control performance. 
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1. Introduction 

The control of discrete systems with time-varying delays has been researched extensively in 
the last few decades. Especially in recent years there are increasing interests in discrete- time 
systems with delays due to the emerging fields of networked control and network 
congestion control (Altman & Basar 1999; Sichitiu et al., 2003; Boukas & Liu 2001). Stability 
problem for linear discrete-time systems with time-delays has been studied in (Kim & Park 
1999; Song & Kim 1998; Mukaidani et al, 2005; Chang et al, 2004; Gao et al, 2004). These 
results are divided into delay-independent and delay-dependent conditions. The delay- 
independent conditions are more restrictive than delay-dependent conditions. In general, 
for discrete-time systems with delays, one might tend to consider augmenting the system 
and convert a delay problem into a delay-free problem (Song & Kim 1998; Mukaidani et 
al.,2005). The guaranteed cost control problem for a class of uncertain linear discrete- time 
systems with both state and input delays has been considered in (Chen et al., 2004). 
Recently, in (Boukas, 2006) new LMI-based delay-dependent sufficient conditions for 
stability have been developed for linear discrete-time systems with time varying delay in the 
state. In these papers above the time-varying delay of discrete systems is assumed to be 
unique in state variables. 

On the other hand, in practice there always exist multiple time-varying delays in state 
variables, especially in network congestion control. Control problems of linear continuous- 
time systems with multiple time-varying delays have been studied in (Xu 1997). Quadratic 
stabilization for a class of multi-time-delay discrete systems with norm-bounded 
uncertainties has been studied in (Shi et al., 2009). 

To the best of author's knowledge, stabilization problem of linear discrete systems has not 
been fully investigated for the case of multiple time- varying delays in state, and this will be 
the subject of this paper. This paper address stabilization problem of linear discrete-time 
systems with multiple time-varying delays by a memoryless state feedback. First, stability 
analysis conditions of these systems are given in the form of linear matrix inequalities 
(LMIs) by a Lyapunov function approach. It provides an efficient numerical method to 
analyze stability conditions. Second, based on the LMIs formulation, sufficient conditions of 
stabilization problem are derived by a memoryless state feedback. Meanwhile, robust 
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stabilization problem is considered based on these formulations and they are numerically 
tractable. 



2. Problem statement 

Considering the dynamics of the discrete system with multiple time-varying time delays as 



fc+i ~ Ax k +2-, A d 



k> fcs [- d max' •••/ 0], 



(1) 



where x k s W" is the state at instant k, the matrices A e'3l' lxn ,A dl ; e 9?"*" are constant 
matrices, (4 represents the initial condition, and d ki are positive integers representing 
multiple time-varying delays of the system that satisfy the following: 

d,<4<d„ i=l,-,N, (2) 

where dj and d { are known to be positive and finite integers, and we let 

d max =max(d i ),i = l,...,N . 

The aim of this paper is to establish sufficient conditions that guarantee the stability of the 
class of system (1). Based on stability conditions, the stabilization problem of this system (1) 
will be handled, too. The control law is given with a memoryless state-feedback as: 

u k =Kx k , X k =<j> k , k = 0,-l,...,-d l , 
where K is the control gain to be computed. 

3. Stability analysis 

In this section, LMIs-based conditions of delay-dependent stability analysis will be 
considered for discrete- time systems with multiple time- varying delays. The following 
result gives sufficient conditions to guarantee that the system (1) for u k = 0, k > is stable. 
Theorem 1: For a given set of upper and lower bounds dj, d t for corresponding time-varying 
delays d h ■, if there exist symmetric and positive-definite matrices P 1 e ■K" X ",Q 1 : e 9?" x " and 



,e9? 


!X ", i=l,...,N and general matrices P 2 


and P 3 such that the following L' 




~ N N 

!=1 1=1 


-A T P 2 - 


-pJa 
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P 2 -PjA 






P 1+ P 3 +P? 
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M = 


~A d iPi 






~A dl P 3 


-Qi 


* 
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-A T d2 P 2 






~ A d2 P 3 





-Qi 




* 




~ A dN P 2 






~^rfN^3 







o -Qn 



<0 (3) 
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Terms denoted by * are deduced by symmetry. Then the system (1) is stable. 
Proof: Consider the following change of variables: 



N 



■-yur ° = -yk+ Ax k+Y, A 



di^k-d,, 



(4) 



i-i 



Define x k = [x[ y[ x[_ dl 
candidate functional: 



■ x k _ dN ] , and consider the following Lyapunov-Krasovskii 
^ t ) = V 1 (* t )+V 2 (* i ) + V 3 (* t ) (5) 



with 

V 1 (x k ) = x T k E T Px k , 

N k-\ 

v 2 (**)=E I xjQ iX} 

,=1 l=k-d ki 

and 

N -rf,+l k-1 

V 3 (x k )=Z Z I *«*<*», 

i=li=-d ; +2m=J:+(-l 

where Q, > and R t > , and E and P are, respectively, singular and nonsingular matrices 
with the following forms: 



I •■■ 

•■■ 



: i ••. : 

■■■ 



, P-- 



P-y ■•• 

P 2 P 3 ••• 

I 

; : '•. i 

••• I 



where P x is a symmetric and positive-definite matrix. 
The difference AV{x k ) is given by 

AV(x k ) = AV 1 (x k ) + AV 2 (x k ) + AV 3 (x k ) 

Let us now compute AV 1 (x t ) : 

AV a (« t ) = V,(i hl ) - V a (* t ) = 5£ E T Px, +1 - x[E r Px, 



(6) 



T 

: y* 



JPiy*-*tJPi*t = y^iy*-2[*t o o - o]p, 



which has the following equivalent formulation using 

N 

o=-yt+A**+ZA«**-4 as 



the fact that 
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The difference AV 2 (*ic) i s given by 

N k N k-1 

AV 2 (x k ) = V 2 (x k+1 )-V 2 (x k )=^ X xjQpci-% X X IQS 



Note that 



N k N k-dj N k-1 N 

Z Z *ta*«=Z Z ^Q^+Z Z *ta*i+Z4Q<.- 

j=l l=k+l-d u i=ll=k+l-d l: i=ll=k+l-cL i=l 



Z Z X ; T Q ! X ( = Z Z x iQi x i + 'Z x k-d u Qi x k-d tl 

1=1 I=Jfc-<J K 1=1 Uk+l-d h 1=1 



Using this, AV 2 (5fc) can be rewritten as 



N k-d. 



^lih) = Z**Qi** - H x I-d u QiXk-d tt + Z Z x JQi x i 

i=l /=1 i=l l=k+l-d ti 

N k-1 N k-1 

+Z Z x lQi x i-H Z x lQi x i- 



=ll=k+l-d, 



For AV s (x k ), we have 



n -i+i k n -i+i k-i 

Av 3 fe)=x Z Z*» R .*m-Z Z Z x ^ x » 

i=ll=-d j +2m=k+l 
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Note that d, < d fo < d, for all i , we get 

N fc— 1 N k-1 N k-dj N k-dj 

X Z *iQi*i*I. Z x / r Qi x /-Z Z *?Q<*/*Z Z x JQi x i 
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(8) 



(9) 
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Finally, by using (7), (8) and (9) together with these inequalities, we obtain 



AV(x k )<[x k y k x k _ dl 



M 



x k-dl 
x k-dN 



<o, 



where 



M 



N N 

1=1 1=1 


-d i )R i +P 1 - 


-A T P 2 - 


-P 2 A 


* 


* 




P 2 -PjA 








P 1 + P 3 +P? 


* 




~ A dl P 2 








~ A dl P 3 


-Qi 




~Aj2^2 








~ A d2?3 







~A dN P 2 








_ -A(iN-P3 






-Q 2 



-Q h 



(10) 



This implies that the system is stable, and then the claim (3) can be established. □ 

Remark: 

As to robust stability analysis of discrete time systems with poytopic-type uncertainties, 

robust stability analysis can be considered by the formulation above. When system state 

matrices in (1) are assumed as 

[A(X{k)) A (B (A(fc))] = 29A fc )[ A > A m\ 5 7 (fc)>0,X5 ; (fc) = l. 
;=i J )=i 

Robust state feedback synthesis can be formulated as: 

For a given set of upper and lower bounds dj, d { for corresponding time-varying delays d ki , 
if there exist symmetric and positive-definite matrices P 1 e s J!" x ",Q ! e 9i" x " and R ; s3?" x ", 
i = l,...,N and general matrices P 2 and P 3 such that the following LMIs hold: 



N 

z 

;'-l 



N 

1 

i-l 



Zft + Z W - W + p i - A J p 2 - p I A j 



P2-P3A, 

~ A dlj P 2 
^ A d2j^2 



~ A dNjP2 



Pl + P 3 +P 3 ' 
~ A d\jP?, 
~ A d2jP?, 



~ A dN]P?, 



-Qi 





-Q2 




-Q A 



< (11) 



;'=i 1. 
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Terms denoted by * are deduced by symmetry. Then the system with poytopic-type 
uncertainties is stable. 

4. Stabilizability 

The aim of this section is to design a memoryless state-feedback controller which stabilizes 
the system (1). When the memoryless state-feedback is substituted with plant dynamics (3), 
the dynamics of closed-loop system is obtained as 



x k+1 ={A + BK)x k +Y i A dl x k _ dh , x k =0 k , k = 0,-l,...,-di 



(12) 



Note that stability analysis condition (3) is not convenient for us to design a memoryless 
state-feedback. By Schur Complememt lemma, equivalent conditions of (3) are given easily 
to solve such a memoryless state-feedback which guarantees the closed-loop system (12) is 
stable. Due to 



-Qi ... 

o -q 2 o ; 

i '•. 

•■■ -Q N 



<o, 



The equivalent formulation of (3) could be obtained as 



N N 



£q,. + £(4 - ^)R, - P 1 - A T P 2 - P T 2 A 



P2-P3A 



P1+P3 + P3 



-pJa 



-P T ,A A 



2 ^Vl L 2 ^d2 

3 An -P3 A d2 



-PlA 



-PlA d 

"^3 AnJ 



-Qi - 

-Q 2 ; 

• '•. 

- -Q N 



~Aa-\ P^ —Aj-i P'. 



Ml 1 2 
W2 



Ajnln A^Lr_ 



■dl l 3 

1 P 

dl L 3 



Aj^tIo Aj-i in 



<0 



If we denote by X the inverse of P , we have 



X: 



X 1 

x 2 x 3 



x 1 =p;\ 



= P 2 X 1+ P 3 X 2 , x 3 =p 3 - 1 



*1=P{ 



Pre- and post multiplying the above LMI, respectively, by X and X and using these 
relations, we will get 
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N N 










Z x iQ< x i+E( d i-4) x i'*< x i 

f=i 1=1 


x 3 + x 3 _ 




P X [X 




~-Qi 


••• 


TO -Aja" 


+ 


' ••• i 


- 


-Q 2 o ; 
o ••. 


-Aj x 









- -Q N \ 


2 - 


A di_ 



<0 



Let S, = Q t and T- = R t , we have 



-x a * 

-AXj - BF + X 2 X 3 + Xj 



X, 



x, 



A, -Ai 






-Si Ail 





-Si 








* 





-S2A2 








-s 2 * 

o •-. 


* 









-SjvAjN 








-S N 






Xi 














-Si 
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di - d-i 







:0 (13) 
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Xi 

X, 



x 3 


-Xi 
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SiAn 





-Si 




SjAff 







-S 













-SivAii 











-Si 





<0 (14) 
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Theorem 2: For a given set of upper and lower bounds dj, d t for corresponding time-varying 

delays d ki , if there exist symmetric and positive-definite matrices X 1 e 3?" x ™ , S ; e s Jt" x " and 

T, e<R' !X ", i=l,...,N and general matrices X 2 and X 3 such that LMIs below hold, the 

memoryless state-feedback gain is given by K = FXj . 

Proof: Now we consider substituting system matrices of (12) into LMIs conditions (13), the 

LMIs-based conditions of the memoryless state-feedback problem can be obtained directly 

as (14). □ 

Remark: 

When these time delays are constant, that is, d = d t = dj , i =1,...,N , theorem 2 is reduced to 

the following condition 



-X, * 

-AX,+X 2 X,+X T , 



x 2 


x, 


*, 


* 






* 





~S\ 4/i 





-s, 



















-s 2 ■■ 

'■ 


* 







~^N^dN 










-s N 




*, 













■ 


s 





X, 























<0 



(15) 



The condition above is delay-independent, which is more restrictive than delay-dependent 

conditions (14). 

Remark: 

When the time-varying delay of discrete systems is assumed to be unique in state variables, 

that is, N = 1 , these results in theorem 2 could be reduced to those obtained in (Boukas, E. 

K. , 2006). 

Remark: 

As to robust control problem of discrete time systems with poytopic-type uncertainties, 

robust state feedback synthesis can be considered by these new formulations. When system 

state matrices in (11) are assumed as 

\A{X{k)) A d ,(X(k)) B(A(fc))] = X9 / (fc)[A / A Mj B ; ], 



9 i (fc)>0,£5 i (fc) = l, 

7=1 

Robust state feedback synthesis can be formulated as: 

For a given set of upper and lower bounds dj, d { for corresponding time varying delays d ki , 
if there exist symmetric and positive-definite matrices X 1 e 3?" x " , S ; e 9i" x " and T ; e 9i" x " , 
i = l,...,N and general matrices X 2 and X 3 such that LMIs (16) hold, the memoryless 
state-feedback gain is given by K = FX^ . 
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5. Numerical example 



d-i — dj 







d« - <ix 



<0 



(16) 



/' = 1 1- 



To illustrate the usefulness of the previous theoretical results, let us give the following 

numerical examples. 

Consider a discrete system with multiple time- varying delays N = 2 as 



1" 

-2 -3 



,A d 



0.01 0.01" 
0.01 



and A, 



0.02 0.25 
0.10 0.01 



with l<d 1 <2,2<d 2 <3. Now the stabilization of this system will be considered with a 

memoryless state feedback. 

Using Matlab LMI toolbox (P. Gahinet, et al., 1995), solving (21) we can get 



X a = 



1.36e-3 4.26e-3" 
4.26e-3 1.62e-2 



X, 



7.31e-3 2.70e-2 
1.95e-2 7.15e-2 



and x, 



-2.73e-4 1.89e-4 
-1.565e-3 -3.42e-3 



Si 



2.17e2 62.5 
62.5 3.45e2 



,S 2 



1.64e2 47.8 
47.8 8.56e2 



4.04e2 1.22e2l 
1.22e2 6.32e2J 



,T 2 



2.84e2 96.6 
96.6 1.03e3 
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Therefore, a memoryless state-feedback gain is given by K = FX{ = [2.0 3.0] . 
The closed-loop discrete-time system with multiple time- varying time delay is simulated in 
case of d-y = 1, d 2 = 2 , d 1 =l, d 2 =3 , d t = 2, d 2 = 2 , and d-y = 2, d 2 = 3 , respectively. And 
these results are illustrated in Figure 1, Figure 2, Figure 3 and Figure 4. These figures show 
that this system is stabilized by the state feedback. 
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roO 
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-0.5 

-1 




5 10 15 Time 20 

Fig. 1. The behavior of the states in case of d 1 =l, d 2 =2 




5 10 15 Time20 

Fig. 2. The behavior of the states in case of d 1 =l, d 2 =3 
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Fig. 3. The behavior of the states in case of d 1 =2, d 2 =2 
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Fig. 4. The behavior of the states in case of d 1 = 2, d 2 = 3 



6. Conclusion 

Stability Criterion and Stabilization for linear discrete-time systems with multiple time- 
varying delays have been considered. Main results have been given in terms of linear matrix 
inequalities formulation. It provided us an efficient numerical method to stabilize these 
systems. Based on these results, it can be also extended to the memory state feedback 
problem of these systems in the future research. 
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1. Introduction 

This chapter is about techniques for robust stability analysis and robust stabilization of 
discrete-time systems with delay in the state vector. The relevance of this study is mainly 
due to the unavoidable presence of delays in dynamic systems. Even small time-delays can 
reduce the performance of systems and, in some cases, lead them to instability. Examples of 
such systems are robotics, networks, metal cutting, transmission lines, chemical and thermal 
processes among others as can be found in the books from Gu et al. (2003), Richard (2003), 
Niculescu (2001) and Kolmanovskii & Myshkis (1999). 

Studies and techniques for dealing with such systems are not new. Since the beginning of 
control theory, researchers has been concerned with this issue, either in the input-output 
approach or in state-space approach. For the input-output approach, techniques such as Pade 
approximation and the Smith predictor are widely used, mainly for process control. The use 
of state space approach allows to treat both cases. For both approaches delays can be constant 
or time-varying. Besides, both the delay and the systems can be precisely known or affected 
by uncertainties. 

In this chapter the class of uncertain discrete-time systems with state delay is studied. For 
these systems, the techniques for analysis and design could be delay dependent or delay 
independent, can lead with precisely known or uncertainty systems (in a polytopic or in a 
norm-bonded representation, for instance), and can consider constant or time-varying delays. 
For discrete-time systems with constant and known delay in the state it is always possible 
to study an augmented delay-free system Kapila & Haddad (1998), Leite & Miranda (2008a). 
However, this solution does not seem to be suitable to several cases such as time-varying delay 
or uncertain systems. 

For these systems, most of the applied techniques for robust stability analysis an robust control 
design are based on Lyapunov-Krasovskii (L-K) approach, which can be used to obtain convex 
formulation problems in terms of linear matrix inequalities (LMIs). 

In the literature it is possible to find approaches based on LMIs for stability analysis, most of 
them based on the quadratic stability (QS), i.e., with the matrices of the Lyapunov-Krasovskii 
function being constant and independent of the uncertain parameters. 

In the context of QS, non-convex formulations of delay-independent type have been proposed, 
for example, in Shi et al. (2003) where the delay is considered time-invariant. In Fridman & 
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Shaked (2005a) and Fridman & Shaked (2005b), delay dependent conditions, convex to the 
analysis of stability and non-convex for the synthesis, are formulated using the approach of 
descriptor systems. These works consider systems with both polytopic uncertainties — see 
Fridman & Shaked (2005a) — and with norm-bounded uncertainties as done by Fridman & 
Shaked (2005b). 

Some other different aspects of discrete-time systems with delayed state have been studied. 
Kandanvli & Kar (2009) present a proposal with LMI conditions for robust stability analysis of 
discrete time delayed systems with saturation. In the work of Xu & Yu (2009), bi-dimensional 
(2D) discrete-time systems with delayed state are investigated, and delay-independent 
conditions for norm-bounded uncertainties and constant delay are given by means of 
nonconvex formulations. In the paper from Ma et al. (2008), convex conditions have been 
proposed for discrete-time singular systems with time-invariant delay. Discrete-time switched 
systems with delayed state have been studied by Hetel et al. (2008) and Ibrir (2008). The 
former establishes the equivalence between the approach used here (Lyapunov-Krasovskii 
functions) and the one used, in general, for the stability of switched systems with time-varying 
delay. The latter gives nonconvex conditions for switched systems where each operation mode 
is subject to a norm-bounded uncertainty and constant delay. 

The problem of robust filtering for discrete-time uncertain systems with delayed state is 
considered in some papers. Delayed state systems with norm-bounded uncertainties are 
studied by Yu & Gao (2001), Chen et al. (2004) and Xu et al. (2007) and with polytopic 
uncertainties by Du et al. (2007). The results of Gao et al. (2004) were improved by Liu et al. 
(2006), but the approach is based on QS and the design conditions are nonconvex depending 
directly on the Lyapunov-Krasovskii matrices. 

The problem of output feedback has attracted attention for discrete-time systems with delay in 
the state and the works of Gao et al. (2004), He et al. (2008) and Liu et al. (2006) can be cited as 
examples of on going research. In special, He et al. (2008) present results for precisely known 
systems with time-varying delay including both static output feedback (SOF) and dynamic 
output feedback (DOF). However, the conditions are presented as an interactive method that 
relax some matrix inequalities. 

The main objective of this chapter is to study the robust analysis and synthesis of discrete-time 
systems with state delay. This chapter is organized as follows. In Section 2 some notations 
and statements are presented, together the problems that are studied and solved in the next 
sections. In sections 3 and 4 solutions are presented for, respectively, robust stability analysis 
and robust design, based in a L-K function presented in section 2. In Section 5 some additional 
results are given by the application of the techniques developed in previous sections are 
presented, such as: extensions for switched systems, to treat actuator failure and to make 
design with pole location. In the last section it is presented the final comments. 



2. Preliminaries and problem statement 

In this chapter the uncertain discrete time system with time-varying delay in the state vector 
is given by 

Cl(ix) ■ { Xk+l = A ^ Xk + A d( a ) x k-d k + B{oc)u k + B w (tx)iv k , 
I z* = C{a)x k + C d {a)x k _ dk + D{oc)u k + D z „{oc)w k , 

where k is the fc-th sample-time, matrices A(a), A d {ix.), B(ct), B w , C{cc), Q(a), D(tx) and D w (oc) 
are time-invariant, uncertain and with adequate dimensions defined in function of the signals 
x k = x(k) G R", the state vector at sample-time k, n k = u(k) 6 R m , representing the control 
vector with m control signals, w k = w(k) 6 R , the exogenous input vector with £ input 
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signals, and z k = z(k) G R p , the output vector with p weight output signals. These matrices 
can be described by a polytope V with known vertices 



V = \ fi(a) G R»+px2«+m+£ . Q( a ) = £ x . n{j aeY 



i=l 



where 



and 



a:J^a i = l,ix i > 0, i 6l[l,N] 

1=1 



O; 



^ ^rf/ 



Q Qi 



B, 



D; 



D„ 



, iel[l,N\. 



The delay, denoted by d k , is supposed to be time-varying and given by: 

rfjt el[d,d],(d,d) GN^ 



(2) 

(3) 
(4) 

(5) 



with d_, d representing the minimum and maximum values of d^, respectively. Thus, any 

system fi(a) G V can be written as a convex combination of the N vertices Q„ i G X[l, N], of 

V. 

The following control law is considered in this chapter: 



u k = Kx k + K d x k _ d . 



(6) 



with [JC|jK(j] G R mx . By replacing (6) in (l)-(4), the resulting uncertain closed-loop system is 
given by 

' x k+1 = A{a)x k + A d {a)x k _ dk + B w (oc)w k 
z k = C{a)x k + C d (cc)x k _ d , + D w {a)w k 



Cl(a) 



(7) 



with n (a) G V, 



V = \ fr(ct) G R n +P x2 "+ e : n(a) = £ UiCli, a G Y 

i=\ 



where 



n, 



A; An 



B„ 



D„ 



(8) 
(9) 

(10) 

Q = q + D,K, C di = C di + DiK d (11) 

Note that, control law (6) requires that both x k and x k _ dk are available at each sample-time. 
Eventually, this can be achieved in physical systems by employing, for instance, a 
time-stamped in the measurements or in the estimated states Srinivasagupta et al. (2004). In 
case of d k is not known, it is sufficient to assume K d = 0. 



. Q c di 

and matrices Aj, A dl , C, e C d i are defined by 

A i = A i + B i K, A dl 



, iei[i,N]. 



B t K d , 
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2.1 Stability conditions 

Since the stability of system Q(a) given in (7) plays a central rule in this work, it is addressed 
in the sequence. Note that, without loss of generality, it is possible to consider the stability of 
the system (7) with w k = 0, V k e N. 
Consider the sequence composed by d + 1 null vectors 

h = i^M 

(rf+l) terms 
In this chapter null initial conditions are always assumed, that is, 

Xk = 4>o,k = fa, kel[-d,o] (12) 

If 'Ptk = Yd' then an equilibrium solution for system (7) with w k = 0, V k G IN, is achieved 
because xjt + i = x k = 0, Vfc > t and a £ O. 

Definition 1 (Uniform asymptotic stability). For a given a. e Y, ffre trivial solution of (7) with 
w k = 0, V fc € N is sfl/d uniformly asymptotically stable if for any k £ R + sue/; that for all initial 
conditions x^ G 0fjj. e <I>j, A: e X[— J,0], ft is verified 



Km ^ t = 0, V)6l[l,rf + 1] 



This allows the following definition: 

Definition 2 (Robust stability). System (7) subject to (3), (5) and (8) is said robustly stable if its 
respective trivial solution is uniformly asymptotically stable V a G Y. 

The main objective in this work is to formulate convex optimization problems, expressed as 
LMIs, allowing an efficient numerical solution to a set of stability and performance problems. 

2.2 Problems 

Two sets of problems are investigated in this chapter. The first set concerns stability issues 
related to uncertain discrete time with time varying delay in the state vector as presented in 
the sequence. 

Problem 1 (Robust stability analysis). Determine if system (7) subject to (3), (5) and (8) is robustly 
stable. 

Problem 2 (Robust control design). Determine a pair of static feedback gains, K and Kj, such that 
(l)-(5) controlled by (6) is robustly stable. 

The other set of problems is related to the performance of the class of systems considered in 
this chapter. In this proposal, the 'Hoo index is used to quantify the performance of the system 
as stated in the following problems: 

Problem 3 CHco guaranteed cost). Given the uncertain system Cl(tx) G V , determine an estimation 
for 7 > such that for all iv k £ £2 there exist z k g £2 satisfying 

\\z k \\ 2 < 7IKII2 (13) 

for all «6Y. In this case, 7 is called an 'H co guaranteed cost for (7). 
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Problem 4 (Robust Hoo control design). Given the uncertain system Ci(a) € P , (1), and a scalar 
7 > 0, determine robust state feedback gains K and K^, such that the uncertain closed-loop system 
Cl(ot) G P, (7), is robustly stable and, additionally, satisfies (13) for all w^ and z^ belonging to ii. 

It is worth to say that, in cases where time-delay depends on a physical parameter (such as 
velocity of a transport belt, the position of a steam valve, etc.) it may be possible to determine 
the delay value at each sample-time. As a special case, consider the regenerative chatter in 
metal cutting. In this process a cylindrical workpiece has an angular velocity while a machine 
tool (lathe) translates along the axis of this workpiece. For details, see (Gu et al., 2003, pp. 2). In 
this case the delay depends on the angular velocity and can be recovered at each sample-time 
k. However, the study of a physical application is not the objective in this chapter. 
The following parameter dependent L-K function is used in this paper to investigate problems 

1-4: 

3 

V(x,k) = £V v (cc,k)>0 (14) 

v=l 
with 

Vi(«,fc) = x' k P(x)x k , (15) 

fc-i 
V 2 (cc,k) = E */Q(*)ty (16) 

j=k-d t 

\-i fc— a 

v 3 ( a ,k) = E E */Q(«o*// ( 17 ) 

e=2-ij=k+e-\ 

The dependency of matrices P(a) and Q(a) on the uncertain parameter a is a key issue on 
reducing the conservatism of the resulting conditions. Here, a linear relation on a is assumed. 
Thus, consider the following structure for these matrices: 

N N 

P( a ) = £>P;; QM = E«,Q; (18) 

1 = 1 1=1 

with a. g Y. Note that, more general structures such as P(a) and Q(a) depending 
homogeneously on a — see Oliveira & Peres (2005) — may result in less conservative 
conditions, but at the expense of a higher numerical complexity of the resulting conditions. 
To be a L-K function, the candidate (14) must be positive definite and satisfy 

AV(oc,k) = V(ot,k + l)-V{a.,k) <0 (19) 

for all I" %l x\_ Ak 1 £ and ft: G Y. 

The following result is used in this work to obtain less conservative results and to decouple 
the matrices of the system from the L-K matrices P(a) and Q(ct). 

Lemma 1 (Finsler's Lemma). Let <p G R", M(x) = M{a) T £ R nx " and Q{a) G R mx " such 
that rank((? (a)) < n. Then, the following statements are equivalents: 

i) cp T M{a)(p < 0, V<p : G{a)(p = 0, <p £ 
ii) g(a) ±T X(ft)6?(ft) ± <0, 
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Hi) 3 /*(«) eR + :M(«)- }i(a)g(a) T g{a) < 

iv) 3 X{oc) G R" Xffl : Al(a) + #(«)£(«) + e(a) T A» T < 

In the case of parameter independent matrices, the proof of this theorem can be found in 
de Oliveira & Skelton (2001). The proof for the case depending on a follows similar steps. 

3. Robust stability analysis and Hco guaranteed cost 

In this section it is presented the conditions for stability analysis and calculation oi Tico 
guaranteed cost for system (7). The objective here is to present sufficient convex conditions 
for solving problems 1 and 3. 



3.1 Robust stability analysis 

Theorem 1. If there exist symmetric matrices < P, ; 6 R" x ", < Q, 6 R" x ", a matrix X 6 

R 3nx ", d k el[djd\ with d and d belonging to N*, such that 



T;= Qi + XBi + BfX 1 <0; 



1,...,N 



with 



Qi 



Pi 

* PQi - Pi o 

* * -Qi 

=d-d+l 



and 



(20) 

(21) 
(22) 
(23) 



Bi= [I-Ai-A,u] 

is verified V a admissible, then system (7) subject to (5) is robustly stable. Besides, (14)-(17) is a 
Lyapunov-Krasovskii function assuring the robust stability of the considered system. 

Proof. The positivity of the function (14) is assured with the hypothesis of P; = Pj > 0, 
Qj = Qj > 0. For the equation (14) be a Lyapunov-Krasovskii function, besides its positivity, 
it is necessary to verify (19) V« 6 fl. From hereafter, the ci dependency is omitted in the 
expressions V v (k), v = 1, ... ,3, To calculate (19), consider 



AVi(fc) = x% +1 P(cL)x k+1 - x T k P^)x k 



(24) 



A--1 



Jfc-1 

AV 2 (k)=x k r Q(a)x k -x k r _ m Q(ot)x k _ ti{k) + E xjQ(a) Xl - £ xfQ(a)x t 

i=k+l-d{k+l) i=k+\-d(k) 



and 

k-d 

AV 3 (k) = (d-d)x k r Q(a)x k - £ x lQ(*) x i 

i=k+l-d 
Observe that the third term in equation (25) can be rewritten as 



(25) 
(26) 



Jfc-l 

E xjQWxi 

i=k+l-d(k+l) 



k-\ 

E xjQ(K)*i- 

i=k+l-d 



k-d 

L 

i=k+l-d(k+l) 



x] Q{&)Xi 



fc-1 k-d 

E xjQ{x)xi+ E" x iQ{*)*i 



(27) 



i=k+l-d(k) 



i=k+l-d 
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Using (27) in (25), one gets 



k-d 



AV 2 (fc) < x[Q(oc)x k - xl_ m Q(a)x k _ d{k) + £ xjQ{u) Xi 

i=k+l-d 
So, considering (24), (26) and (28) the following upper bound for (19) can be obtained 

AV(fc) < x T k+l P(a)x k+1 + xJ[j3Q(a) - P(«)]x t - xl_ d{k) Q{a)x k _ d(k) < 

Taking into account (7) and using Lemma 1 with 

<p = £k= [ x I+i 4 x Ld k \ 

'P(oc) 



M(u) 



G(a) 





J8Q(«)-P(«) 
* -Q(«) 

;i-a(«) -a*(«)] 



then (29) is equivalent to 

Y(«) = M{a)+X{<x)g{ot) + g{ct) T X{a) J < 0. 
which is assured whenever (20) is verified by taking X(a) = X , 

N N 

« 6 Y, Qj and Z?, given in (21) and (23), respectively, completing the proof. 



(28) 

(29) 

(30) 

(31) 
(32) 

(33) 

(34) 

□ 



An important issue in Theorem 1 is that there is no product between the matrices of the system 
and the matrices of the Lyapunov-Krasovskii proposed function, (14). This can be exploited 
to reduce conservatism in both analysis and synthesis methods. 

Example 1 (Stability Analysis). In this example the stability analysis condition given in Theorem 1 
is used to investigate system (7), with D w = 0, where 



0.6 
0.35 0.7 



and A 



di 



0.1 
0.2 0.1 



(35) 



This system has been investigated by Liu et al. (2006), Boukas (2006) and Leite & Miranda (2008a). 

The objective here is to establish the larger delay interval such that this system remains stable. The 

results are summarized in Table 1. 

Although Theorem 1 and the condition from Liu et al. (2006) achieve the same upper bound for d k , the 

L-K function employed by Liu et al. (2006) has 5 parts while Theorem 1 uses a function with only 3 

parts, as given by (14)-(17). 

Consider that (35) is affected by an uncertain parameter being described by a polytope (8) with A\ and 

Aj\ given by (35) and Ai = 1.1A\ and A^ = l.lAfa. In this case the conditions ofBoidcas (2006) 

are no longer applicable and those from Liu et al. (2006) are not directly applied, because of type of 

the system uncertainty. Using Theorem I it is possible to assure the robust stability of this system for 

l4+i -41 < 3. 
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Condition 


d 


I 


Boukas (2006) [Theorem 3.1] 


2 


10 


Liu et al. (2006) 


2 


13 


Theorem 1 


2 


13 



Table 1. Maximum delay intervals such that (7) with (35) is stable. 

3.2 Estimation of %co guaranteed cost 

Theorem 2 presented in the sequel states a convex condition for checking if a given 7 is an Hoo 
guaranteed cost for system (7). 

Theorem 2. If there exist symmetric matrices < P; € R" x ", < Q; G R" x ™, a matrix Xfi 6 
R3n+p+fx n+ V/ ^ k e jy^ ^ witn ^ an ^ j belonging to N*, and a scalar u = j 2 e R+, such that 



V-Hi = Qu, + X H B m + B H ]X H T < 0, i = l,...,N, 



with 



Q 



Hi 



Qi 

• -yip 



(36) 



(37) 



where Qj is given by (21) and 



Bm 



(38) 



B { B u 

[0Q C dl ] -ID„ 

wrc'fft £>, g-roen by (23), f/;eH system (7) subject to (5) with null initial conditions, see (12), is robustly 
stable with an Hoo guaranteed cost given by 7 = ^/~fi. Besides, (14)-(17) is a L-K function assuring 
the robust stability of the considered system. 

Proof. Following the proof given for Theorem 1, it is possible to conclude that the positivity of 
(14) is assured with the hypothesis of P(a) = P(ct) T > 0, Q(ot) = Q{oc) T > and, by (29) that 



AV(k) < x k+1 P(a)x M + 4[/3Q(«) - P(x)]x k - 4-d(k)Q(<*) x k-d(k) < 



(39) 



Consider system (7) as robustly stable with null initial conditions given by (12), assume /( = J 2 
and signals w k and z k belonging to 1%. In this case, it is possible to verify that V(tx, 0) = and 
V(a, 00) approaches zero, whenever w k goes to zero as k increases, or to a constant cf> < 00, 
whenever w k approaches (p < 00 as k increases. Also, consider the 'Hco performance index 
given by 



z\z k - uwlwk\ 



/(*,*) = E k 

/c=0 
Then, using (39), /(a, k) can be over bounded as 

J(a,k) < £ \z[z k - uw T k w k + kV(0L,k)\ 
k=0 

00 r 1 

< E [ z I z k - H^I w k + 4+i p O)*it+i + 4[0Q(«) - p (*)} x k - x ' k -d{k)Q( a ) x k-d(k)\ 



(40) 



k=0 
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which can be rewritten as 

CO 

7(*,*)<£tfe«(«)& (4i) 

fc=0 

T N 

with £ fc = [£ T z[ w\\ , l k defined in (30), and Q%(u) = £ <hQhu <* S Y. Then, applying 

i=\ 
Lemma 1 to 

with M (a) = Qft(a), <p = £*, 



tfQ n (*Kk<0 subject to (7), (42) 



and «6Y, (42) is equivalent to 



<B(«) B w (a) 

~0C{ct) Q(«)] -ID„,(d) 



(43) 



Y H (a) = g^(«) + ^( a )^M + S„( a ) J *„(«)' <0. (44) 

Once (36) is verified, (44) is assured with the special choice X^{cc) = X n e r3h+p+£xh+ p — 
i.e., eliminating the dependency on the uncertain parameter a — and noting that Q(a) = 
Y^Li^i^Hi' convexity is achieved, and (36) can be used to recover (44) by Y%(a) = 
Y4L1 OLi^Hb a € Y. Thus, this assures the negativity of ](ol, k) for all w k G ^2 implying that (7) 
is robustly stable with T-Lco guaranteed cost given by 7 = ^/Jl. □ 

In case of time-varying uncertainties, i.e. a. = aj- = a.(k), the conditions formulated in both 
Theorem 1 and Theorem 2 can be adapted to match the quadratic stability approach. In this 
case, it is enough to use P, = P, Q, = Q, i € I[l, N]. This yields conditions similar to (20) 
and (36), respectively, with constant L-K matrices. See Subsection (5.1) for a more detailed 
discussion on this issue. 

Note that, it is possible to use the conditions established by Theorem 2 to formulate the 
following optimization problem that allows to minimize the value of }i = 7 : 

imin ]i 

Pi>0;Qi>0;X (45) 

such that (36) is feasible 

4. Robust Woo feedback design 

The stability analysis conditions can be used to obtain convex synthesis counterpart 
formulations for designing robust state feedback gains K and Kj, such that control law (6) 
applied in (1) yields a robustly stable closed-loop system, and, therefore, provides a solution 
to problems 2 and 4. In this section, such conditions for synthesis are presented for both robust 
stabilization and robust "Hco control design. 

4.1 Robust stabilization 

The following Theorem provides some LMI conditions depending on the difference d — d to 
design robust state feedback gains K and Kj that assure the robust stability of the closed-loop 
system. 
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Theorem 3. If there exist symmetric matrices < P, ■ e R" x ", < Q, G R" 



,N, 



matrices J6R" x ",We R'" x " and W d e R mx ", d k e l[d,d] with d and d belonging to N*, such 
that 

-p t + T + J T -(AiT + B.W) -(A dl T + B { W d )~ 

%= * ISO,--?; o 

* * -Qi 

are verified with & given by (22), then system (l)-(3) is robustly stabilizable with (6), where the robust 
static feedback gains are given by 



< 0, 



1,...,N 



(46) 



K = WJ 7 " 
yielding a convex solution to Problem 2. 



and K d = W d T 



-l 



(47) 



Proof. Observe that, if (46) is feasible, then J- is regular, once block (1, 1) of (46) assures P; + 
J- + J-" r < 0, allowing to define 

T = I 3 <8> J- J (48) 

Then, by replacing W and W d by KJ- and K d J- , respectively obtained from (47), it is possible 
to recover T, = T¥,T T < with X = [ T~ r 0] T , P, ■ = F^P^- 1 , Q, = F^Q,?- 1 and 
the closed-loop system matrices A; = ( A + BjK) and A d { = (A d j + BjK d ) replacing Aj and A d j 
in (20), which completes the proof. □ 

Note that, conditions in Theorem 3 encompass quadratic stability approach, since it is always 
possible to choose P, = P and Q; = Q, i = 1, . . . , N. Also observe that, if d^ is not available 
at each sample-time, and therefore x^_ d cannot be used in the feedback, then it is enough to 
choose W d = leading to a control law given by U; c = Kx^. Finally, note the convexity of the 
conditions stated in Theorem 3. This is a relevant issue, once most of the results available in 
the literature depend on a nonlinear algorithm to solve the stabilization problem. 

Example 2 (Robust Stabilization). Consider the discrete-time system studied in Leite & Miranda 
(2008a) with delayed state described by (1) with D w = and 



1 
-2 -3 



0.01 0.1 
0.1 



(49) 



Suppose that this system is affected by uncertain parameters \p\ < 0.07, \Q\ < 0.1 and \n\ < 0.1, such 
that 

A(p) = (l + p)A; A d (8) = (l + e)A d ; B(n) = (l + n)B (50) 

These parameters yield a polytope with 8 vertices determined by the combination of the extreme values 
of p, 8 and n. Also, suppose that delay is not available on line and it is bounded as 1 < d^ < 10. By 
applying the conditions presented in Theorem 3 with W d = — this is a necessary issue once the delay 
value is not known at each sample-time — it is possible to get the robust stabilizing gain 



K 



1.9670 2.7170 



(51) 



The behavior of the states of the closed-loop response of this uncertain discrete-time system with 
time-varying delay is shown in Figure 4. It has been simulated the time response of this system at 
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0.5 - 




x 2 (k) 




Fig. 1. The behavior of the states X\ (k) (top) and X2W (bottom), with dj- 6 I[l, 10] randomly 
genereated and the robust state feedback gain (51). 

each vertex of the polytope that defines the uncertain closed-loop system. The initial conditions have 
been chosen as 



1 
-1 



1 
-1 



11 terms 
and the value of the delay, d/ c , has been varied randomly. Please see Leite & Miranda (2008a) for details. 
In Figure 4, it is illustrated the stability of the uncertain close-loop system, assured by the robust state 
feedback gain (51). 



4.2 Robust "Hoo feedback design 

An stabilization condition assuring the 'Hoo cost of the feedback system is stated in the sequel. 

Theorem 4. If there exist symmetric matrices < P, G R" x ",0 < Q, 6 W xn , matrices T € R nx ", 
W e R mx ", W d e R mx ", a scalar variable 6 e]0, 1] and for a given u = y 2 G R+ such that 

'Pi 



J - F 1 A t J + B,W A di T + BjW d 







■k 
* 



PQt - Pi 




-Qi 

-k 
-k 



Tr,T 



T l C{ +W l D 
F T Cl + WjDl 

-ei 

* 






-pi 



< 0, i = 1, . . . , N (52) 



are feasible with B given by (22), then system (l)-(3) is robustly stabilizable with (6) assuring a 
guaranteed Hoo cost given by 7 to the closed-loop system by robust state feedback gains K and K d 
given by (47). 

Proof. To demonstrate the sufficiency of (52), firstly note that, if it is verified, then the 
regularity of T is assured due to its block (1,1) that verifies P, — J- — J- < 0. Besides, there 
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exist a real scalar k £]0,2[ such that for 9 6]0,1], k(k — 2) = —6. Thus, replacing block (4,4) of 
(52) by k(k — 2)Ip, the optimization variables W and Wj by KJ- and K^J- ' , respectively, using 
the definitions given by (10)-(11) and pre- and post-multiplying the resulting LMI by 7% (on 
the left) and by Tl, (on the right), with 



T H 



T 
G 
* 1/ 



-i -l 



(53) 



with T given by (48) and G £ W y P, it is possible to obtain Y%; < 0, with t m given by 



¥ 



Hi 



F- T P l F- 1 - J- 1 - J- x J- T (Ai + B t K) 



F- T (Ad, + B,K d ) 



* -k 
■k -k 

* k 



-F-'Qi? 



-l 



FBw 



(cf 


+ K l Dl 


)G> 







Q 






(Cd< 


+ 4», 


'') G T 







Cm 




G( 


k 2 - 2;c) 


G> 


GDw 




* 




-V-h 



(54) 



Observe that, assuming G 



-Ip, block (4, 4) of (54) can be rewritten as 



G ( k 2 - 2k) G t 



k 2 - 2k 



1-- N I 



Ip - -Ip 

In + G + G 



1 

K r 

T 



(55) 



assuring the feasibility of W ni < given in (36) with P; = F'^iT' 1 , Q, = T~' QjJ 7 ', 

i el[l,N],and 



X H 



r^- 1 


0" 

















G 









completing the proof. 



□ 
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Theorem 4 provides a solution to Problem 4. This kind of solution can be efficiently achieved 
by means of, for example, interior point algorithms. Note that all matrices of the system can 
be affected by polytopic uncertainties which states a difference w.r.t. most of the proposals 
found in the literature. Another remark concerns the technique used to obtain the synthesis 
condition: differently from the usual approach for delay free systems, here it is not enough 
to replace matrices in the analysis conditions with their respective closed-loop versions and 
to make a linearizing change of variables. This makes clear that the Tica control of systems 
with delayed state is more complex than with delay free systems. Also, note that the design 
of state feedback gains K and K^ can be done minimizing the guaranteed 'H C o cost, 7 = ^JyZ, 
of the uncertain closed-loop system. In this case, it is enough to solve the following convex 
optimization problem: 



mm 
Pi > 0; Q, • > 0; < 6 < 1; 

W; W d ; F 
such that 



/' 



(52) is feasible 



(56) 



Example 3 (Hco Design). A physically motivated problem is considered in this example. It consists of 
a fifth order state space model of an industrial electric heater investigated in Chu (1995). This furnace is 
divided into five zones, each of them with a thermocouple and a electric heater as indicated in Figure 2. 
The state variables are the temperatures in each zone (x\,.. .,x^), measured by thermocouples, and 
the control inputs are the electrical power signals (u.\, . . . , u$) applied to each electric heater. The 




X\ > ' X 2 v ' X3 * ' A"4 v ' x 5 

Fig. 2. Schematic diagram of the industrial electric heater. 

temperature of each zone of the process must be regulated around its respective nominal operational 
conditions (see Chu (1995) for details). The dynamics of this system is slow and can be subject to several 
load disturbances. Also, a time-varying delay can be expected, since the velocity of the displacement of 
the mass across the furnace may vary. A discrete-time with delayed state model for this system has been 
obtained as given by (1) with d^ = d = 15, where 

0.97421 0.15116 0.19667 -0.05870 0.07144 ' 
-0.01455 0.88914 0.26953 0.11866 -0.22047 

0.06376 0.12056 1.00049 -0.03491 -0.02766 (57) 

-0.05084 0.09254 0.28774 0.82569 0.02570 

0.01723 0.01939 0.29285 0.03544 0.87111 
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irfO 



-0.01000 -0.08837 -0.06989 
0.02363 0.03384 0.05282 
-0.04468 -0.00798 0.05618 
-0.04082 0.01153 -0.07116 
-0.02537 0.03878 -0.04683 



0.18874 0.20505 
-0.09906 -0.00191 
0.00157 0.03593 
0.16472 0.00083 
0.05665 -0.03130 



(58) 



B = B n 



0.53706 -0.11185 0.09978 0.04652 0.25867 
-0.51718 0.73519 0.57518 0.40668 -0.12472 

0.29469 0.31528 1.16420 -0.29922 0.23883 
-0.20191 0.19739 0.41686 0.66551 0.11366 
-0.11835 0.16287 0.20378 0.23261 0.36525 



(59) 



and C = D = I5, Q = 0, D w = 0, B w = 0.11 with Aq, A^, and Bq being the nominal matrices of 
this system. Note that, this nominal system has unstable modes. The design of a stabilizing state 
feedback gain for this system has been considered in Chu (1995) by using optimal control theory, 
designed by an augmented delay-free system with order equal to 85 and a time-invariant delay d = 15, 
by means of a Riccati equation. 

Here, robust Hoa state feedback gains are calculated to stabilize this system subject to uncertain 
parameters given by \p\ < 0.4, \n\ < 0.4 and \o~\ < 0.08 that affect the matrices of the system as 
follows: 

A(p) = A(l + p), A d (6) = A d (l + e), B(a) = B(l + a) (60) 

This set of uncertainties defines a polytope with 8 vertices, obtained by combination of the upper and 
lower bounds of uncertain parameters. Also, it is supposed in this example that the system has a 
time-varying delay given by 10 < d^ < 20. 

In these conditions, an %«, guaranteed cost 7 = 6.37 can be obtained by applying Theorem 4 that 
yields the robust state feedback gains presented in the sequel. 



K : 



K, 



-2.2587 -1.0130 -0.0558 0.4113 0.9312 
-2.0369 -2.1037 0.0822 1.5032 0.0380 
0.9410 0.5645 -0.7523 -0.8688 0.3801 
-0.5796 -0.2559 0.0454 -1.0495 0.4072 
-0.0801 0.4106 -0.4369 0.5415 -2.4452 _ 

" -0.0625 0.2592 0.0545 -0.2603 -0.5890 

-0.1865 0.1056 -0.0508 0.1911 -0.4114 

0.1108 -0.0460 -0.0483 -0.0612 0.1551 

0.0309 0.0709 0.1404 -0.3511 -0.1736 

0.0516 -0.1016 0.1324 -0.0870 0.1158 



(61) 



(62) 



5. Extensions 

In this section some extensions to the conditions presented in sections 3 and 4 are presented. 

5.1 Quadratic stability approach 

The quadratic stability approach is the source of many results of control theory presented in 
the literature. In such approach, the Lyapunov matrices are taken constant and independent of 
the uncertain parameter. As a consequence, their achieved results may be very conservative, 
specially when applied to uncertain time-invariant systems. See, for instance, the works 
of Leite & Peres (2003), de Oliveira et al. (2002) and Leite et al. (2004). Perhaps the main 
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advantages of the quadratic stability approach are the simple formulation — with low 
numerical complexity — and the possibility to deal with time-varying systems. In this case, all 
equations given in Section 2 can be reformulated by using time-dependency on the uncertain 
parameter, i.e., by using oc = a k . In special, the uncertain open-loop system (1) can be 
described by 



Cl v (ix k ) 



x k+l = A(a k )x k + A d (a k )x k _ dk + B(oc k )u k + B w {ot k )w k , 
Zjfc = C(a k )x k + C d (a k )x k _ d + D(a k )u k + D w (a k )w k , 



with a k G Y v 



Y B 



«*:£<% = !, a kl >0, i€l[l,N] 



(63) 



(64) 



which allows to define the poly tope V given in (2) with a k replacing a. Still considering control 
law (6), the resulting closed-loop system is given by 



Cl v (a k ) 



x k+1 = A{a k )x k + A d (oc k )x k _ dk + B w {ot k )w k , 
Zk = C{a k )x k + C d {a k )x k _ dt + D l0 (a k )w k , 



(65) 



with Cl(tx k ) £ V given in (8) with a k replacing a. 

The convex conditions presented can be simplified to match with quadratic stability 
formulation. This can be done in the analysis cases by imposing P t ■ = P > and Q, ■ = Q > 
in (20) and (36) and, in the synthesis cases, by imposing P,- = P > 0, Q; = Q > 0, i = 1, . . . ,N. 
This procedure allows to establish the following Corollary. 

Corollary 1 (Quadratic stability). The following statements are equivalent and sufficient for the 
quadratic stability of system Cl v {ct k ) given in (65): 

i) There exist symmetric matrices < P e R" x ", < Q G R nx ", matrices F e R" x ", G G R" x " 
and H G R" x " G R" x ", d k G X[d,d] with d and d belonging to N*, such that 



1 n 



P + F T + F G T - FA { 

* PQ-P- AjG T - GAi 



H T - FA di 

~AjH T - G T A dl 

(Q + HA dl + A T dl H J 



<0, 



(66) 



is verified for i = 1, . . . , N. 

ii) There exist symmetric matrices < P G R" x ", < Q G R" x ", d k G l[d,d~] with d and d 
belonging to N*, such that 



3>; 



is verified for i = 1, . . . , N. 



AjPA, + PQ-P AjPA d , 

A T dt PA dl - Q 



< 



(67) 



Proof. Condition (66) can be obtained from (20) by imposing P t ■ = P > and Q, = Q > 0. 
This leads to a Lyapunov-Krasovskii function given by 

fc 1 1-rf fc 1 

V(x k )=x' k Px k + £ x' j Qx j + £ X] x 'jQ x i 
j=k-d(k) e=2-dj=k+e-i 
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which is sufficient for the quadratic stability of Cl v (a. k ). This condition is not necessary for 
the quadratic stability because this function is also not necessary, even for the stability of the 
precisely known system. The equivalence between (66) and (67) can be stated as follows: i) => 
ii) if (66) is verified, then (67) can be recovered by <£,- = Tj^fqiTqi with 



r qi 



qi i qi 

At A d 



hn 



i) <= ii) On the other hand, if (67) is verified, then it is possible by its Schur 's complement to 
obtain 

--P PA, PA di - 

* PQ-P 

* * -Q. 

which assures the feasibility of (66) with F = — P, G = H = 0, completing the proof. □ 



<t>, < <=> 



< 0, i = l,...,N (68) 



It is possible to obtain quadratic stability conditions corresponding to each of the formulations 
presented by theorems 2, 3, 4 following similar steps of those taken to obtain Corollary 1. 
However, due to the straight way to obtain such conditions, they are not shown here. 
Nevertheless, quadratic stability based conditions may lead to results that are, in general, 
more conservative than those achieved by similar formulations that employ parameter 
dependent Lyapunov-Krasovskii functions. 

5.2 Actuator failure 

Partial or total actuator failures are important issues on real word systems and the 
formulations presented in this chapter can also be used to investigate the robust stability as 
well as to design robust state feedback control gains assuring stability and 'Hco guaranteed 
performance for the uncertain closed-loop system under such failures. The robustness against 
actuator failures plays an important role in industry, representing not only an improvement 
in the performance of the closed-loop system, but also a crucial security issue in many plants 
Leite et al. (2009). In this case, the problem of actuator failures is cast as a special type of 
uncertainty affecting the input matrix B, being modeled as Bp(t), with p(t) G X[0, 1]. If 
p(t) = 1, then the actuator is perfectly working. On the other hand, when the value of p(t) is 
reduced, it means that the actuator cannot delivery all energy required by the control law. The 
limit is when p(t ) = 0, meaning that the actuator is off. Once the actuator failure implies on 
time-varying matrix B(a), i.e., B(a. k ), it is necessary to employ quadratic stability approach, 
as described in subsection 5.1. 

5.3 Switched systems with delayed state 

Another class of time-varying systems is composed by the discrete-time switched systems 
with delay in the state vector. In this case the system can be described by 

x k+l = A(a k )x k + A d {a k )x k _ dt + B(oc k )u(oc k ) (69) 

with adequate initial conditions and the uncertain parameter a. k = a(k) is 

«w-{5:5&£ <™> 
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and o\ is an arbitrary switching function defined as 

cr k : N ->■ X[l, N] (71) 

where N is the number of subsystems. The matrices [A(a k )\A d (a. k )\B(ix k )] e ^nx2n+m aie 
switched matrices depending on the switching function (71) and can be written as the vertices 
of the polytope defined by the set of submodes of the system. Naturally, except the vertices, 
no element of this polytope is reached by the system. Therefore, function o\ can select one of 
the subsystems [^4|^4rf|B],-, i = 1, . . . , N, at each instant k. Those definitions can be done with 
all other matrices presented in (69) or (65). 
It is usual to take the following hypothesis when dealing with switched delay systems: 

Hypothesis 1. The switching function is not known a priori, but it is available at each sample-time, 
k. 

Hypothesis 2. All matrices of system (69) (or mutatis mutandis (65)) are switched simultaneously 
by (71). 

Hypothesis 3. Both state vectors, x k and x k _ dj _, are available for feedback. 

These hypotheses can be considered on both stabilization and %«, control problems proposed 
in sections 3 and 4. An important difference w.r.t. the main stabilization problems investigated 
in this chapter is that, if a k is known, it is reasonable to use also a switched control law given 
by 

Uk = K(a k )x k + K d (cc k )x k _ dk (72) 

where the gains K(a k ) and K d (a k ) are considered to stabilize the respective subsystem i, i = 
1, . . . , N, and assure stable transitions cr k — Y o~ k+ \. Thus, the switched closed-loop system may 
be stabilizable by a solution of this problem, being written as in (65) with 

A(a k ) = A(a k ) + B(oc k )K(a k ) A d (a k ) = A d (a k ) + B(oc k )K d (oc k ) (73) 

The stability of the closed-loop system can be tested with the theorem presented in the sequel. 

Theorem 5. If there exist symmetric matrices < P; eR" x ",0< Q,- G R" x ", matrices F,- G R" x ", 
G, G ]R" X " and H, G R" x ", i = 1, . . . , N, and a scalar & = d — d + 1, with d and d known, such 
that 



Pj + Fj + Fi Gj - F,A, Hj - F t A dt 

BQt - Pi - AjGj - G t A, -AjHj - G,A dl 

-(Qe + H t A dl + A T di Hj)i 



< 0, (74) 



di i 

for (i,j, £) G X[l, N] x X[l, N] x X[l, N], then the switched time-varying delay system (69)-(73) with 
u k = is stable for arbitrary switching function a k . 

As it can be noted, a relevant issue of (74) is that the extra matrices are also dependent on the 
switching function o~ k . This condition can be casted in a similar form of (20) as follows 

Y<ri,j,t = Qi,j,e + X t B, + BjxJ < 0, (i,j, i) G X[l, N] x X[l, N] x X[l, N] (75) 

where 

Qi,u 

The synthesis case, i.e. to solve the problem of designing K{ and K d j, i = 1, . . . , N, such that 
the (69)-(72) is robustly stable, is presented in the following theorem. 



'Pi 










* 


m- 


P, 





* 


-k 




-Qtl 
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Theorem 6. If there exist symmetric matrices < P; G R" x ", < Q; G R" x ", matrices F; G R nx ", 
W, eR" x ' and W,;,' G R" x ^, f = 1,...,N, and a scalar B = d — d + 1, with d and d known, such 
that 

" Pj + Ti + Tj - ( Af Ti + B^) - (A dh Fi + B^ di ) " 
%= * PQi-Pi 0_ <0, (76) 

for (i,j,£) G X[l, N] x X[l, N] x X[l, N], then the switched system with time-varying delay (69) is 
robustly stabilizable by the control law (72) with 



K,- = W,F7 



and K 



di 



W dl F, 



(77) 



The proof of theorems 5 and 6 can be found in Leite & Miranda (2008b) and are omitted here. 
An important issue of Theorem 6 is the use of one matrix Xj for each submode. This is possible 
because of the switched nature of the system that reaches only the vertices of the polytope. 

Example 4. Consider the switched discrete-time system with time varying delay described by (69) 
where where A(a k ) = A„ + {-l^pL'J, A d {<r k ) = (0.225 + (-l) (7l 0.025)A„ and 



B(a k ) = [01.5 01.5]' +(-1)^(0 0.5 0.5]' 



with 



A n 



0.8 -0.25 1 
10 
0.2 0.03 
10 



(78) 



L = [0, 0, 1, 0]',/ = [0.8, -0.5, 0, l],(T k G {1, 2},p = 0.35. This system with 2 submodes has been 
investigated by Leite & Miranda (2008b). Note that, even for d = d = 1, conditions from Theorem 5 
fail to identify this system as a stable one. Observe that, once the delay is time-varying, conditions 
presented in Montagner et al. (2005), Phat (2005) and Yu et al. (2007) cannot be applied. Supposing 
d = 1, a search on d has been done to find its maximum value such that the considered system is 
stabilizable. Two alternatives are pursued: firstly, consider that only x k is available for feedback, i.e., 
Kj = 0. Conditions of Theorem 6 are feasible until d = 15, for which value it is possible to determine 
the following gains: 

K Th6,i = [0.1215 0.0475 -1.6326 -0.4744] 

K Th6,2 = [-0.1494 0.1551 -0.8168 -0.5002] 

Secondly, consider that both x k and xjt-d t are available for feedback. By using Theorem 6 it is possible 
to stabilize the switched system for 1 < d k < 335. In this case, with d = 335, conditions of Theorem 6 
lead to 



K 2 
K-di 

Kd2 



[-0.6129 0.3269 -1.2873 -1.1935] 
[-0.2199 0.1107 -0.6450 -0.4890] 
[-0.1291 0.0677 -0.3228 -0.2685] 
[-0.0518 0.0271 -0.1291 -0.1076] 



(79) 
(80) 
(81) 
(82) 



These gains are used in a numerical simulation where random signals for a k G {1, 2} and for 1 < 
d(k) < 335 have been generated as indicated in Figure 3. The initial condition used in this simulation 
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Fig. 3. The switched function, o~(k) and the varying delay, d^ 
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336 terms 
Thus, it is expected that the delayed state degenerate the overall system response, at least for the first 
d = 335 samples, since it is like an impulsive state action arrives at each sample instant for < k < 
335. Note that, this initial condition is harder than the ones usually found in the literature. The state 
behavior of the switched closed-loop system with time-varying delay is presented in Figure 4. Observe 
that the initial value of the state are not presented due to the scale choice. As can be noted by the response 
behavior presented in Figure 4, the states are almost at the equilibrium point after 400 samples. The 
control signal is presented in Figure 5. In the top part of this figure, it is shown the control signal part 
due to K aii x(k) and in the bottom the control signal due to K^ x(k — d^). The actual control signal is, 
thus, the addition of these two signals. If quadratic stability is used in the system of this example, the 
results are more conservative as can be seen in Leite & Miranda (2008b). 



5.4 Decentralized control 

It is interesting to note that the synthesis conditions proposed in this chapter, i.e. theorems 3, 4, 
6 as well as the convex optimization problem S-u^, can be easily used to design decentralized 
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Fig. 4. The behaviors of the states x lk to x ik , with 1 < d(k) < 335 (see Fig. 3). 

control gains. This kind of control gain is usually employed when interconnected systems 
must be controlled by means of local information only. In this case, decentralized control gains 
K = Kd and K d = K d ^ can be obtained by imposing block-diagonal structure to matrices W, 
W d and J- as follows 

W = W D = block-diag{ W 1 , . . 

W d = W dD = block-diag{W],. 



T = Td = block-diag{ T , 






where Q denote the number of defined subsystems. In this case, it is possible to get robust 



block-diagonal state feedback gains K^ 



\N D F D l a 



dD 



W dD J r D . It is worth to 



mention that the matrices of the Lyapunov-Krasovskii function, P(oc) and Q(oc), do not have 
any restrictions in their structures, which may leads to less conservative designs. 

5.5 Static output feedback 

When only a linear combination of the states is available for feedback and the output signal is 
given by i/; c = Cx; c , it may be necessary to use the static output feedback. See the survey made 
by Syrmos et al. (1997) on this subject. In case of C with full row rank, it is always possible 
to find a regular matrix L such that CL~ 
transformation applied to (1) it yields 



[iplo]. Using such matrix L in a similarity 



h+1 = M«)x k + A d (a)x k _ d + B{oc)u k , 



(83) 
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Fig. 5. Control signal Wj- = K^x^ + K^JC^n , with -K^Xjt and i^d,a k x k-d k shown in the top 
and bottom parts, respectively. 

where A{ot) = LA(a.)L^ 1 , A^{tx) = LAd((x)L~ l and 8(a) = LB(ix), f; c = Lx; c and the output 
signal is given by j/j. = [iplu] X/ c . Thus, the objective here is to find robust static feedback 
gains K G W /l and K, d eW xl such that (83) is robustly stabilizable by the control law 



u k = K-Vk + K-dVk-d k 



(84) 



These gains can be determined by using the conditions of theorems 3, 4, 6 with the following 
structures 



T 



J 







" / - 



w = [w K \o] 



w d 



[w^|o] 



with F X G W X P, 7" 21 G R("-P) X P, Tf- G R("-P) x ("-P), W k G W xn , W Kd G RP X " which 
yields 

K= [fC\0] and K d = [/Q|u] 

Note that, similarly to the decentralized case, no constraint is taken over the 
Lyapunov-Krasovskii function matrices leading to less conservative conditions, in general. 

5.6 Input delay 

Another relevant issue in Control Theory is the study of stability and stabilization of input 
delay systems, which is quite frequent in many real systems Yu & Gao (2001), Chen et al. 
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(2004). In this case, consider the controlled system given by 

x k+1 = A{ot)x k + B(a)u k _ dk (85) 

with A(a) and B(a) belonging to polytope (2), A di = and a e Y. In Zhang et al. (2007) this 
system is detailed investigated and the problem is converted into an optimization problem 
in Krein space with an stochastic model associated. Here, the delayed input control signal is 
considered as 

Uk-d t = K d x k _ dk (86) 

The closed-loop-system is given by 

x/c+i = A(oi)x k + A d (a)x k _ dk (87) 

with A(a) = A(a), A d (a) = B(a)K d . Thus, with known K d , closed-loop system (87) is 
equivalent to (7) with null exogenous signal w k . This leads to simple analysis stability 
conditions obtained from Theorem 1 replacing A; by Aj and A d j by B{K d , i = 1, . . . , N. 
Besides, similar replacements can be used with conditions presented in theorems 2 and 5 and 
in Corollary 1. The possibility to address both controller fragility and input delay is a side 
result of this proposal. In the former it is required that no uncertainty affects the input matrix, 
i.e., B(u) = B, Ma. 6 Y, while the latter can be used to investigate the bounds of stability 
of a closed-loop system with a delay due to, for example, digital processing or information 
propagation. 

In case of the design of K d it is possible to take similar steps with conditions of theorems 3, 4 
and 6. In this case, it is sufficient to impose, A d { = 0, z = 1, . . . ,N and W = that yield K = 0. 
Finally, observe that static delayed output feedback control can be additionally addressed here 
by considering what is pointed out in Subsection 5.5. 

5.7 Performance by delay-free model specification 

Some well developed techniques related to model-following control (or internal model 
control) can be applied in the context of delayed state systems. The major advantage of 
such techniques for delayed systems concerns with the design with performance specification 
based on zero-pole location. See, for example, the works of Mao & Chu (2009) and Silva 
et al. (2009). Generally, the model-following control design is related to an input-output 
closed-loop model, specified from its poles, zeros and static gain, from which the controller 
is calculated. As the proposal presented in this chapter is based on state feedback control, 
it does not match entirely with the requirements for following-model, because doing state 
feedback only the poles can be redesigned, but not the zeros and the static gain. To develop a 
complete following model approach an usual way is to deal with output feedback, that yields 
a non-convex formulation. One way to match all the requirements of following model by 
using state feedback and maintaining the convexity of the formulation, is to use the technique 
presented by Coutinho et al. (2009) where the model to be matched is separated into two 
parts: One of them is used to coupe the static gain and zeros of the closed loop system with 
the prescribed model and the other part is matched by state feedback control. Consider the 
block diagram presented in Figure 6. In this figure, O(a) is the system to controlled with 
signal u k . This system is subject to input io k which is required to be reject at the output y k . 
Please, see equation (1). fi m stands for a specified delay-free model with realization given by 

The model receives the same exogenous input of the system to be controlled, w k , 

and has an output signal y mk at the instant k. 



A m 


B m 


_ wn 


D m _ 
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Fig. 6. Following model inspired problem. 

The objective here is to design robust state feedback gains K and K d to implement the control 
law (6) such that the 'Hoo guaranteed cost between the input w k and the output e k = y k — y m j is 
minimized. In other words, it is desired that the disturbance rejection of the uncertain system 
with time-varying delay in the state have a behavior as close as possible to the behavior of 
the specified delay-free model Q m . The dashed line in Figure 6 identifies the enlarged system 
required to have its T-Lcx, guaranteed cost minimized. 
Taking the closed-loop system (7) and the specified model of perturbation rejection given by 



x mk+l 
Vmk 






«7> 



(88) 
(89) 



where x mk £ R" m is the model state vector at the /c-th sample-time, y mk 6 R p is the output 



of the model at the same sample-time and w k 6 R is the same perturbation affecting the 
controlled system, the difference e k = y mk — z k is obtained as 



e k = [ C„, -(C(«) + D(a)K) -(Q(«) + D(«)K d ) ] 



Xfnk 
Xk 

x k-d k 



+ [D m -D w (oc)]w k (90) 

Thus, by using (1) with (88)-(89) and (90) it is possible to construct an augmented system 
composed by the state of the system and those from model yielding the following system 



Cl(a) 



x k+1 = A(oc)x k + A d {u)x k _ dt + B w (a)w k 
e k = C{a)x k + C d {tx)x k _ dk + D l0 {cc)w k 



(91) 
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with 4= [x T mk x T k '_ 



G R"»'+", O(a) G V, 

A(«) g R»+»»+px2(»+«,„)+^ . n (a) 



N 

1 = 1 



a G Y 



(92) 



where 



n, 



[A 


A«a 


B-wi 


Lq 


Qi 


D wl 



A, n 






A, + B;K 



lC n -(Q + DiK) 





A di + Bj^rf 



- (Q, + D,-K d 



Urn L^7i 



6l[l,N]. 



(93) 



Therefore, matrices in (93) — A,, Am, B wi , Cj, Q;, D ; „, — can be used to replace their respective 
in (38) and (23). As a consequence, LMI (36) becomes with 3(w + n m ) + 2(p + £) rows. Since 
the main interest in this section is to design K and K d that minimize the 'Hco guaranteed 
cost between e^ and w^, only the design condition is presented in the sequel. To achieve 
such condition, similar steps of those taken in the proof of Theorem 4 are taken. The main 
differences are related to i) the size and structure of the matrices and ii) the manipulations 
done to keep the convexity of the formulation. 



Theorem 7. If there exist symmetric matrices < P, 
eR n+n '» xn+n >», matrices J 7 



Qm Qm 
* Q22; 
matrix, W g RP X ", W d g R?' x " 



g ]R"+' 1 »i xn +"»i A g ]R" X "™ 



-^11 F\2 
J22A J" 2 2 

a scfl/ar variable 8 g]0, 1] and for a given }i 



Pm Pm 

* -P22, 



G R 



n+R«XH+R. 



, < Q; 



is a given 



7 2 sue/; tftsi 



Y; 



P 12; -^ 12 -A J ^ 



fit/ _ -^11 _ -^11 
* 



(A di F22 + B t W d )A A dl F 22 + BiW d 





—Qm — Ql2i 

* -Q22; 



T-pT 
22 



A m J-- 



m->~\\ 



A„,F- 



m-r\2 



Pm - ?n - Hi (A1J22 + BiW)A A { J 22 + B,W 



* 




PQw - 


Pm PQl2i - 
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* 




• 


PQzu - 
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-01 
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■k 




-V-h 

i = 1, 



< 



• , N (94) 



then system (l)-(5) is robustly stabilizable by (6) with 

K = WT 22 and K d = W d J r 22 l 



(95) 
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providing an Hco guaranteed cost 7 
signal Wfc 



y/pi between the output ej-, as defined by (93), and the input 



Proof. The proof follows similar steps to those of the proof of the Theorem 4. Once (94) is 

-^11 F\2 
J22 A J 2 2 



verified, then the regularity of J- 



is assured by the block 



Ps-F- F 1 



P\\i - T\i - J r 11 P\2i - F\2 - A J^j 



tT 

22 



< 0. 



* ^22; _ -^22 - ?z. 

Thus it is possible to define the congruence transformation 7% given by (53) with 



r = i 3 <g> t- 



h 



J 12 A 722 



-T 



to get % = T%f{TS- In block (7, 7) of %, it always exist a real scalar k g]0,2[ such that for 
6 G]0, 1], k(k — 2) = —9. Thus, replacing this block by k(k — 2)I„, the optimization variables 
W and Wrf by KJ-22 and K^J-22, respectively, and using the definitions given by (91)-(93) it 
is possible to verify (36) by i) replacing matrices A,, A di , Q, Q,-, B w i and Dj„; by Aj, A di , Q, 
Q,-, B w i and D a „, respectively, given in (93); ii) choosing G = \\ p that leads block (7, 7) to be 
rewritten as in (55); Hi) assuming 



-T 



Q, 



and 



-7"11 -7"12 


— 1 


Plli Pl2i 


Tn 


Tia 


-7"22 A -F22 
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. * Pzx. 


^22 A ^22 


•7"11 -^12 


1 ~ T 


Qm Qm 


•^11 -^12 


-7"22 A ^22 




* Q22t. 


. -T^A ^22 




Jll F\2 


-1 







J22A ^22 




* H = 


















K h 






-1 



-1 



which completes the proof. 



□ 



An important aspect of Theorem 7 is the choice of A G ]R" X "»> m (94). This matrix plays 
an important role in this optimization problem, once it is used to adjust the dimensions of 
block (2, 1) of J- that allows to use J-22 to design both robust state feedback gains K and K^. 
This kind choice made a priori also appears in some results found on the literature of filtering 
theory. Another possibility is to use an interactive algorithm to search for a better choice of A. 
This can be done by taking the following steps: 



Set max iteri 



maximum number of iterations; j ' < — 0; e =precision; 



Choose an initial value of A 



A such that (94) is feasible. 



(a) Set }ij < — p; Ap < — pf, T 2 2,j < — -^22; Wj 



W;W, 



d,j 



w d 



3. While (A// > e)AND(j < maxjter) 
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(a) set;<— ; + l; 

(b) If /is odd 

i. Solve (94) with ^22 
ii. Set A i — Ay; 
Else 

i. Solve (94) with A <- 



Tll,y W 



W f , W d 



W, 



i.y 



A; 



ii. SetJ 7 - 



22,7 



End_if 

(c) Set fij <- 
End while 



T 22 ; Wj «— W; W d// 

f; a f< — I(F/-F/-i)I; 



w d 



4. Calculate K and K^ by means of (95); 

5. Set f(* = py 

Once this is a non-convex algorithm — only steps 3.(b).i are convex — different initial guesses 
for A may lead to different final values for the controllers K and K^, as well as to the 7 = ,/F* 
To overcome the main drawback of this proposal, two approaches can be stated. The first 
follows the ideas of Coutinho et al. (2009) by designing an external loop to the closed-loop 
system proposed in Figure 6. In this sense, it is possible to design a transfer function that can 
adjust the gain and zeros of the controlled system. The second approach is based on the work 
of Rodrigues et al. (2009) where a dynamic output feedback controller is proposed. However, 
in this case the achieved conditions are non-convex and a relaxation algorithm is required. 
In the example presented in the sequel, Theorem 7 with 



A 



In. 



n — n m xn m 



(96) 



Example 5. Consider the uncertain discrete-time system with time-varying delay dj. G X[2, 13] as 
given in (1) with uncertain matrices belonging to polytope (2)-(3) with 2 vertices given by 





"0.6 ' 
0.35 0.7 


/ A& = 


"0.1 " 
0.2 0.1 


Bwi — 




1 


, Bi = 


"0" 
1 


, B 


C 1= [10], 


C dl = [0 0.05], 




D„,i = 





2, Dj = C 


).l, 


D w 



1.1A1 



U2 



1.1A 



d\ 



Jwl 



LIB 



ivli 



B 2 = LIB 



(97) 

(98) 

(99) 

(100) 

It is desired to design robust state feedback gains for control law (6) such that the output of this 
uncertain system approaches the behavior of delay-free model given by 



C 2 = l.lCj, Q 2 = 1-lQi 

= l.lD ro i 02 = 1.10! 



n„ 



G(z) 



0.1847z - 0.01617 
z + 0.3 



-0.3 



-0.2864 



0.25 



0.1847 



(101) 



Thus, it is desired to minimize the Hx, guaranteed cost between signals e^ and w^ identified in Figure 6. 
The static gain of model (101) was adjusted to match the gain of the controlled system. This procedure 
is similar to what has been proposed by Coutinho et al. (2009). The choice of the pole and the zero was 
arbitrary. Obviously, different models result in different value ofHco guaranteed cost. 
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By applying Theorem 7 to this problem, with A given in (96), it has been found an Fioo guaranteed cost 
7 = 0.2383 achieved with the robust state feedback gains: 

K= [1.8043 -0.7138] and K d = [-0.1546 -0.0422] (102) 

In case of unknown rfj-, Theorem 7 is unfeasible for the considered variation delay interval, i.e., imposing 
K d = 0. On the other hand, if this interval is narrower, this system can be stabilized with an "Hoo 
guaranteed cost using only the current state. So, reducing the value of d from d = 13, it has been found 
that Theorem 7 is feasible for d^ 6 Z[2, 10] with 

K= [-2.7162 -0.6003] and K d = (103) 

and 7 = 0.3427. Just for a comparison, with this same delay interval, if K and K d are designed, then 
the T-Lco guaranteed cost is reduced about 37.8% yielding an attenuation level given by 7 = 0.2131. 
Thus, it is clear that, whenever the information about the delay is used it is possible to reduce the 
Hoa guaranteed cost. Some numerical simulations have been done considering gains (102), and a 
disturbance input given by 

(0, if k = 0ork> 11 ,.„., 

^={l, ifl<10 ( 104 > 

Two conditions were considered: i) d/ c = 13, VA: < and different values of a-y G [0, 1]; and ii) 
d^ = d =G X{2, 13] with a.\ = 1 (i.e., only for the first vertex). The output responses of the controlled 
system have been performed with d k = 13, Vfc > 0. This family of responses and that of the reference 
model are shown at the top of Figure 7 with solid lines. A red dashed line is used to indicate the desired 
model response. The absolute value of the error (|e^| = |yj- — y m k\) is shown in solid lines at the 
bottom of Figure 7 and the estimate Fica guaranteed cost provide by Theorem 7 in dashed red line. The 
respective control signals are shown in Figure 8. 

The other set of time simulations has been performed using only vertex number 1 (a\ = 1). In this 
numerical experiment, the perturbation (104) has been applied to system defined by vertex 1 and twelve 
numerical simulations were performed, one for each constant delay value dj- = d G [2, 13]. The results 
are shown in Figure 9: at the top, a red dashed line indicates the model response and at the bottom it is 
shown the absolute value of the error (|ej.| = \y k — y m ^\) in solid lines and the estimate 'H 00 guaranteed 
cost provide by Theorem 7 in dashed red line. This value is the same provide in Figure 7, once it is the 
same design. The respective control signals performed in simulations shown in Figure 9 are shown in 
Figure 10. 

At last, the frequency response considering the input w^ and the output e^ is shown in Figure 11 with 
a time-invariant delay. For each value of delay in the interval [2,13] and a e [0,1], a frequency 
sweep has been performed on both open loop and closed-loop systems. The gains used in the closed-loop 
system are given in (102). It is interesting to note that, once it is desired that y^ approaches y m ^, i.e., 
e^ approaches zero, the gain frequency response of the closed-loop should approaches zero. By Figure 11 
the FLoo guaranteed cost of the closed-loop system with time invariant delay is about 0.1551, but this 
value refers to the case of time-invariant delay only. The estimative provided by Theorem 7 is 0.2383 
and considers a time varying delay. 

6. Final remarks 

In this chapter, some sufficient convex conditions for robust stability and stabilization 
of discrete-time systems with delayed state were presented. The system considered is 
uncertain with polytopic representation and the conditions were obtained by using parameter 
dependent Lyapunov-Krasovskii functions. The Finsler's Lemma was used to obtain LMIs 
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Vk 0.1 





Fig. 7. Time behavior of i/; c and |ej.| in blue solid lines and model response (top) and 
estimated 'Hco guaranteed cost (bottom) in red dashed lines, for d^ = 13 and a £ [0, 1] 
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Fig. 8. Control signals used in time simulations presented in Figure 7. 

condition where the Lyapunov-Rrasovskii variables are decoupled from the matrices of the 
system. The fundamental problem of robust stability analysis and stabilization has been dealt. 
The %oo guaranteed cost has been used to improve the performance of the closed-loop system. 
It is worth to say that even all matrices of the system are affected by polytopic uncertainties, 
the proposed design conditions are convex, formulated in terms of LMIs. 
It is shown how the results on robust stability analysis, synthesis and on Wrx, guaranteed cost 
estimation and design can be extended to match some special problems in control theory such 
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Fig. 9. Time behavior of y^ and \e^\ in blue solid lines and model response (top) and estimated 
■Hoo guaranteed cost (bottom) in red dashed lines, for vertex 1 and delays from 2 to 13. 
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Fig. 10. Control signals used in time simulations presented in Figure 9. 

as decentralized control, switched systems, actuator failure, output feedback and following 
model conditions. 

It has been shown that the proposed convex conditions can be systematically obtained by 
i) defining a suitable positive definite parameter dependent Lyapunov-Krasovskii function; 
ii) calculating an over bound for AV(k) < and Hi) applying Finsler's Lemma to get a set 
of LMIs, formulated in a enlarged space, where cross products between the matrices of the 
system and the matrices of the Lyapunov-Krasovskii function are avoided. In case of robust 
design conditions, they are obtained from the respective analysis conditions by congruence 
transformation and, in the "Hco guaranteed cost design, by replacing some matrix blocs by 
their over bounds. Numerical examples are given to demonstrated some relevant aspects of 
the proposed conditions. 
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1.5 2 

a»[rad/s] 

Fig. 11. Gain frequency response between signals e/ c and iffy for the open loop (top) and 
closed-loop (bottom) cases for delays from 2 to 13 and a sweep on a G [0, 1] . 

The approach used in this proposal can be used to deal with more complete 
Lyapunov-Krasovskii functions, yielding less conservative conditions for both robust stability 
analysis and design, including closed-loop performance specifications as presented in this 
chapter. 
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1. Introduction 

Uncertainties in a control system may be the results modeling errors, measurement errors, 
parameter variations and a linearization approximation. Most physical dynamical systems 
and industrial process can be described as discrete time uncertain subsystems. Similarly, the 
unavoidable computation delay may cause a delay time, which can be considered as time- 
delay in the input part of the original systems. The stability of systems with parameter 
perturbations must be investigated. The problem of robust stability analysis of a nominally 
stable system subject to perturbations has attracted wide attention (Mori and Kokame, 1989). 
Stability analysis attempts to decide whether a system that is pushed slightly from a steady- 
state will return to that steady state. The robust stability of linear continuous time-delay 
system has been examined (Su and Hwang, 1992; Liu, 2001). The stability analysis of an 
interval system is very valuable for the robustness analysis of nominally stable system 
subject to model perturbations. Therefore, there has been considerable interest in the 
stability analysis of interval systems (Jiang, 1987; Chou and Chen, 1990; Chen, 1992). 
Time-delay is often encountered in various engineering systems, such as the turboject 
engine, microwave oscillator, nuclear reactor, rolling mill, chemical process, manual control, 
and long transmission lines in pneumatic and hydraulic systems. It is frequently a source of 
the generation of oscillation and a source of instability in many control systems. Hence, 
stability testing for time-delay has received considerable attention (Mori, et al., 1982; Su, et 
al., 1988; Hmamed, 1991). The time-delay system has been investigated (Mahmoud, et al., 
2007; Hassan and Boukas, 2007). 

Grey system theory was initiated in the beginning of 1980s (Deng, 1982). Since then the 
research on theory development and applications is progressing. The state-of-the-art 
development of grey system theory and its application is addressed (Wevers, 2007). It aims 
to highlight and analysis the perspective both of grey system theory and of the grey system 
methods. Grey control problems for the discrete time are also discussed (Zhou and Deng, 
1986; Liu and Shyr, 2005). A sufficient condition for the stability of grey discrete time 
systems with time-delay is proposed in this article. The proposed stability criteria are simple 
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to be checked numerically and generalize the systems with uncertainties for the stability of 
grey discrete time systems with time-delay. Examples are given to compare the proposed 
method with reported (Zhou and Deng, 1989; Liu, 2001) in Section 4. 

The structure of this paper is as follows. In the next section, a problem formulation of grey 
discrete time system is briefly reviewed. In Section 3, the robust stability for grey discrete 
time systems with time-delay is derived based on the results given in Section 2. Three 
examples are given to illustrate the application of result in Section 4. Finally, Section 5 offers 
some conclusions. 

2. Problem formulation 

Considering the stability problem of a grey discrete time system is described using the 
following equation 



x(k + l) = A(®)x(k) 



(1) 



where x(k) s R" represents the state, and A(®) represents the state matrix of system (1). 

The stability of the system when the elements of A(®) are not known exactly is of major 

interest. The uncertainty can arise from perturbations in the system parameters because of 

changes in operating conditions, aging or maintenance-induced errors. 

Let <8>,-.- (i,j = 1,2, ...,n) of A(®) cannot be exactly known, but ® ( - are confined within the 

intervals e <; < ® ; - < fy . These e, and/,, are known exactly, and <§>,•■ e ®,® . They are called 

white numbers, while ®« are called grey numbers. A(®) has a grey matrix, and system (1) 

is a grey discrete time system. 

For convenience of descriptions, the following Definition and Lemmas are introduced. 

Definition 2.1 

From system (1), the system has 



A(®) = [®«U 



®11 ®12 "■ ®1« 

® 21 ® 22 ■■■ ® 2 „ 



® , ® , •■■ ® 



(2) 



^ L e ij\nxn 



(3) 



t-J li Inxn 



fu In 

JZl J22 
fnl Jn2 



fin 
fin 

J nn 



(4) 



where E and F represent the minimal and maximal punctual matrices of A(®) , respectively. 
Suppose that A represents the average white matrix of A(®) as 
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and 



e H + fv 



E + F 



(5) 



A G =[a gij ] nxn =[® iraij ] nxn =A(®)-A (6) 

M = [m ij \ mn =[f ir a ij \ nxn = -F-A (7) 

where A G represents a bias matrix between A((8>) and A; M represents the maximal bias 
matrix between F and A. Then we have 

\A G \„<\M\ m (8) 

where |M| ra represents the modulus matrix of M; r\M] represents the spectral radius of 

matrix M; I represents the identity matrix, and A(M) is the eigenvalue of matrix M. This 

assumption enables some conditions to be derived for the stability of the grey discrete 

system. Therefore, the following Lemmas are provided. 

Lemma 2.1 (Chen, 1984) 

The zero state of x(k + 1) = Ax(k) is asymptotically stable if and only if 

|det(zI-A)|>0, for |z|>l. 

Lemma 2.2 (Ortega and Rheinboldt, 1970) 

For any hxii matrices R, T and V, if \R\ m < V , then 

a. r[R]<r[\R\ m ]<r[V] 

b. r[RT]<r[\R\ m \T\ m )<r[V\T\ m ] 

c. r[R + T]<r[\R + T\ m ]<r[\R\ m + \T\ m ]<r[V + \T\ m ]. 

Lemma 2.3 (Chou, 1991) 

If G(z) is a pulse transfer function matrix, then 

\G(z)\ m <X\G(K)\ m ^H(C(K)), for |z|>l, 

where G(K) is the pulse-response sequence matrix of the multivariable system G(z) . 

Lemma 2.4 (Chen, 1989) 

For an n x n matrix R, if r[R] < 1 , then |det(I ± R)\ > . 

Theorem 2.1 

The grey discrete time systems (1) is asymptotically stable, if A(<8>) is an asymptotically 

stable matrix, and if the following inequality is satisfied, 

r[H(G(K))\M\ m ]<l (9) 

where H(G(K)) and [ML are defined in Lemma 2.3 and equation (8), and G(K) is the pulse- 
response sequence matrix of the system 
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G(z) = (zl-Ay 1 
Proof 
By the identity 

det[RT] = det[R]det[T], 
for any two nxn matrices R and T, we have 

\det[zI-A(®)]\ = \det[zI-{A + A G )]\ = \det[I-(zI-Ay 1 {A G )] | Idet[zl-A] | (10) 

Since A represents an asymptotically stable matrix, then applying Lemma 2.1 clearly shows 
that 

|det[zl - A] > 0| , for |z|>l (11) 

If inequality (9) is satisfied, then Lemmas 2.2 and 2.3 give 

r[{zI-A)-\A G )] = r[G(z)(A G )] < r[\C(z)\ m |A G |J 

^[|G(z)|jM| m ] (12) 

<r[H(G(K))\M\ m ] 
< 1, for \z\ > 1 
From equations (10)-(12) and Lemma 2.4, we have 

|det[zl - A(®)]| = |det[zl - (A + A G )]| 

= |det[f-(zJ , -A)" 1 (A G )]||det[2J'-A]|>0, for |z|>l. 

Hence, the grey discrete time system (1) is asymptotically stable by Lemma 2.1. 

3. Grey discrete time systems with time-delay 

Considering the grey discrete time system with a time-delay as follows: 

x(k + 1) = A,(®)x(/c) + Bi(®)x(k - 1) (13) 

where Ai(®) and Bi(®) denotes interval matrices with the properties as 

Aim = [®5U and B,(®) = [®yU (14) 

where «J < ®"^ < a| and bj < ®| < b| . 
Indicate 

Ai = [4lnx»/ A2 = [a}j] nxn , Bi = [blj] nxn , B 2 = [bfj]„ x „ ■ (15) 

and let 

A = [. L= !i!4. = (^ (16a) 
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and 

B= [ ^u = [b i^=fcM (16b) 

where A and B are the average matrices between A 1 and A 2 ,B 1 and B 2 , respectively. 
Moreover, 

AA m = [<]»*« =[®5-« # ] W(B =A I (®)-A (17a) 

and 

AB m =[^]„ xn =[®|-b ii ]„ xn =B I (®)-B (17b) 

where AA ra and AB m are the bias matrices between A 7 and A , and B ; and B , 
respectively. Additionally, 

M x = [mj]„ x „ = \a\ - a it ] nx „ =A 2 -A (18a) 

and 

Ni = [n}j\ mn = [bfj - b y ]„ x „ = B 2 - B (18b) 

where Mi and Ni are the maximal bias matrices between Aj and A, and B2 and B, 
respectively. Then we have 

|AA m | „< |M 1 |,„ and |AB m | „< |Ni|,„ . (19) 

The following theorem ensures the stability of system (13) for all admissible matrices 

A,AA m ,B and AB ra with constrained (19). 

Theorem 3.1 

The grey discrete time with a time-delay system (13) is asymptotically stable, if nominal 

system A 7 (<8>) is an asymptotically stable matrix, and if the following inequality is satisfied, 

r[H(G d (K))(\Mi\ m + \B\ m +\Ni\ m )] < 1 (20) 

where H(Gd(K)) are as defined in Lemma 2.3, and Gj(K) represents the pulse-response 
sequence matrix of the system 

G d (z) = (zl-Ay 1 
Proof 
By the identity 

det[RT] = det[R]det[T], 
for any two nxn matrices R and T, we have 

\det[zI-(A I (®) + B I ('S>)z- 1 )] | = |det[zJ , -(A + AA m +(B + AB m )z" 1 )] I 

1 , ( 21 ) 

= det[I-(zI-A)- 1 (AA ra +(B + AB ffl )z- 1 )] |det[zI-A] | 
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Since A is an asymptotically stable matrix, then applying Lemma 2.1 clearly shows that 

|det[zl - A]\ > , for |z|>l 
If inequality (20) is satisfied, then Lemmas 2.2 and 2.3 give 



(22) 



^(zI-A)" 1 ! 



AA„,+(B + AB, 



,)z- 1 )] = r[G,(z)(AA ra+ (B + AB ffl )2- 1 )] 



<r 
<r 
<r 
<r 
<r 
<r 



G d (z)\ 

C d (z)\ 

GM 

\\Gi(z)\ 
[|G rf (z)| 

[H(G,(X))(|M 1 
<1, /or \z\ >1 



AA m +(B + AB m )z- 1 | ) 
AAj +|(B + AB m )z- 1 | ) 

' m I I m / 

AAj + |(B + ABj| |z -1 | ) 

">\m |V m 'lm| \ m ] 

AA m | m+ |(B + AB m )|J] 
AB «L)' 



(23) 



AAJ +B + 



m I \m 



Equations (21)-(23) and Lemma 2.4 give 



|det[zl - (Aj(®) + B^z- 1 )] | = |det[zl - (A + AA,„ + (B + ABJz" 1 )] | 

= |det[I-(zI-A)" 1 (AA m +(B + AB m )z" 1 )]| \det[zl -A]\>0, for |z|>l 

Therefore, by Lemma 2.1, the grey discrete time with a time-delay system (13) is 
asymptotically stable. 

4. Ilustrative examples 

Example 4.1 

Consider the stability of grey discrete time system (1) as follows: 



x(k + l) = A(®)x(k), 



where 



A(®) = 



®11 ®12 
®21 ®22 



with -0.5 < ®f a < 0.5, 0. 1 < ®? 2 < 0.8, -0.3 < ®f a < 0.2,-0.4 < ®f 2 < 0.5 . 
From equations (2)-(5), the average matrices is 

0.45 
-0.05 0.05 
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and from equations (6)- (7), the maximal bias matrix Mis 

M= 



0.5 0.35 
0.25 0.45 



By Lemma 2.3, we obtain 



Then, the equation (9) is 



H(G(K))= 



1.0241 0.4826 
0.0536 1.0725 



r[H(G(K))\M\ m ] = 0.9843 < 1. 

Therefore, the system (1) is asymptotically stable in terms of Theorem 2.1. 

Remark 1 

Zhou and Deng (1989) have illustrated that the grey discrete time system (1) is 

asymptotically stable if the following inequality holds: 



p(k)<l 



(24) 



By applying the condition (24) as given by Zhou and Deng, the sufficient condition can be 

obtained as p(k) = 0.9899 < 1 to guarantee that the system (1) is still stable. 

The proposed sufficient condition (9) of Theorem 2.1 is less conservative than the condition 

(24) proposed by Zhou and Deng. 

Example 4.2 

Considering the grey discrete time with a time-delay system (Shyr and Hsu, 2008) is 

described by (13) as follows: 



x(k + 1) = Ai(®)x(k) + Bi(®)x(k - 1) 



where 



with 



and 



A,(®) = 



®21 ®22j 



,B,(®) = 



®21 ®22j 



-0.2 < ®?, < 0.2, -0. 2 < ®?, < 0.1, -0.1 < ®1 < 0.1,-0.1 < ®o ? < 0.2 



0.1 < ®5i ^ 0.2, 0.1 < ®? 2 < 0.2,0.1 < ®2j < 0.15, 0.2 < ®\ 2 < 0.25. 



Equation (15) and (25) give 



Ar 



0.2 


-0.2" 




"0.2 0.1" 




r 0.1 0.1" 




0.1 


-0.1 


/ A 2 = 


0.1 0.2 


/ Bi= 


0.1 0.2 


/ B 2 = 



0.2 0.2 
0.15 0.25 



(25) 
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From equations (16), the average matrices are 

A= 



-0.05 
0.05 



0.15 0.15 
0.125 0.225 



and from equations (18), the maximal bias matrices Mi an d Ni are 



Mr 



0.2 0.15 
0.1 0.15 



Nr 



0.05 0.05 
0.025 0.025 



0.0526 0.0526 
1.0526 



By Lemma 2.3, we obtain 

H(G d (K))= 
From Theorem 3.1, the system (13) is stable, because 

>'[H(G rf (K))(|M 1 | m + \B\ m +\ Nl \j] = °- 4462 < 1- 

Example 4.3 

Considering the grey discrete time-delay systems (Zhou and Deng, 1989) is described by 
(13), where 



®11 ®?2 
0921 0^22 



,(«) = 






with -0.24 < ®? a < 0.24, 0.12 < ® a 12 < 0.24, -0.12 < ® a 21 < 0.12, 0.12 < ®" 22 < 0.24 and 
0.12 < ®5i < 0.24, 0.12 < ®? 2 < 0.24, 0.12 < ®^ < 0.18, 0.24 < ®\ 2 < 0.30. 



Equation (15) and (25) give 



At 



-0.24 0.12 
-0.12 0.12 



,Ar 



0.24 0.24" 
0.12 0.24 



,Bi= 



0.12 0.12 
0.12 0.24 



,B 2 = 



From (16)-(18), we obtain the matrices 



0.18 
0.18 



Mi 



0.24 0.06 
0.12 0.06 



0.18 0.18 
0.15 0.27 



,Ni 



0.24 0.24 
0.18 0.30 



0.06 0.06 
0.03 0.03 



0.5459 0.3790 
0.3659 0.4390 



By Lemma 2.3, we obtain 

H(G d (K))= 
From Theorem 3.1, the system (13) is stable, because 

r[H(G,(K))(|M 1 | m+ \B\ „+|N 1 | m )] = 0.8686 < 1 
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According to Theorem 3.1, we know that system (13) is asymptotically stable. 

Remark 2 

If the following condition holds (Liu, 2001) 

min max £ ( e tj + f tj ) , max £ ( e jt + f jt ) < 1 (26) 

{ ' 7=1 ' 7=1 J 

then system (13) is stable i,j = 1,2, ...,n , where 

E = [ey], e u = of, e tj = max{|®?-|,|o||} for i * j 
and 



The foregoing criterion is applied in our example and we obtain 



min | max]£(e,y +/,?)/ max^(e J1 +/ |1 ) = 1.02 > 1 

{ ' 7=1 ' 7=1 J 

which cannot be satisfied in (26). 

5. Conclusions 

This paper proposes a sufficient condition for the stability analysis of grey discrete time 
systems with time-delay whose state matrices are interval matrices. A novel sufficient 
condition is obtained to ensure the stability of grey discrete time systems with time-delay. 
By mathematical analysis, the stability criterion of the proposed is less conservative than 
those of previous results. In Remark 1, by mathematical analysis, the presented criterion is 
less conservative than that proposed by Zhou and Deng (1989). In Remarks 2, by 
mathematical analysis, the presented criterion is to be less conservative than that proposed 
by Liu (2001). Therefore, the results of this paper indeed provide an additional choice for the 
stability examination of the grey discrete time time-delay systems. The proposed examples 
clearly demonstrate that the criteria presented in this paper for the stability of grey discrete 
time systems with time-delay are useful. 
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1. Introduction 

This article is focused on analyzing stability and Ci gain properties for switched systems 
composed of a family of linear discrete-time descriptor subsystems. Concerning descriptor 
systems, they are also known as singular systems or implicit systems and have high abilities 
in representing dynamical systems [1, 2]. Since they can preserve physical parameters in the 
coefficient matrices, and describe the dynamic part, static part, and even improper part of 
the system in the same form, descriptor systems are much superior to systems represented 
by state space models. There have been many works on descriptor systems, which studied 
feedback stabilization [1, 2], Lyapunov stability theory [2, 3], the matrix inequality approach 
for stabilization, H2 and/or Tica control [4-6]. 

On the other hand, there has been increasing interest recently in stability analysis and design 
for switched systems; see the survey papers [7, 8], the recent books [9, 10] and the references 
cited therein. One motivation for studying switched systems is that many practical systems 
are inherently multi-modal in the sense that several dynamical subsystems are required 
to describe their behavior which may depend on various environmental factors. Another 
important motivation is that switching among a set of controllers for a specified system can be 
regarded as a switched system, and that switching has been used in adaptive control to assure 
stability in situations where stability can not be proved otherwise, or to improve transient 
response of adaptive control systems. Also, the methods of intelligent control design are based 
on the idea of switching among different controllers. 

We observe from the above that switched descriptor systems belong to an important class of 
systems that are interesting in both theoretic and practical sense. However, to the authors' 
best knowledge, there has not been much works dealing with such systems. The difficulty 
falls into two aspects. First, descriptor systems are not easy to tackle and there are not rich 
results available up to now. Secondly, switching between several descriptor systems makes 
the problem more complicated and even not easy to make clear the well-posedness of the 
solutions in some cases. 

Next, let us review the classification of problems in switched systems. It is commonly 
recognized [9] that there are three basic problems in stability analysis and design of switched 
systems: (i) find conditions for stability under arbitrary switching; (ii) identify the limited 
but useful class of stabilizing switching laws; and (iii) construct a stabilizing switching law. 
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Specifically, Problem (i) deals with the case that all subsystems are stable. This problem 
seems trivial, but it is important since we can find many examples where all subsystems are 
stable but improper switchings can make the whole system unstable [11]. Furthermore, if 
we know that a switched system is stable under arbitrary switching, then we can consider 
higher control specifications for the system. There have been several works for Problem (i) 
with state space systems. For example, Ref. [12] showed that when all subsystems are stable 
and commutative pairwise, the switched linear system is stable under arbitrary switching. 
Ref. [13] extended this result from the commutation condition to a Lie-algebraic condition. 
Ref. [14, 15] and [16] extended the consideration to the case of £2 g am analysis and the case 
where both continuous-time and discrete-time subsystems exist, respectively. In the previous 
papers [17, 18], we extended the existing result of [12] to switched linear descriptor systems. 
In that context, we showed that in the case where all descriptor subsystems are stable, if the 
descriptor matrix and all subsystem matrices are commutative pairwise, then the switched 
system is stable under impulse-free arbitrary switching. However, since the commutation 
condition is quite restrictive in real systems, alternative conditions are desired for stability of 
switched descriptor systems under impulse-free arbitrary switching. 

In this article, we propose a unified approach for both stability and £2 g am analysis of 
switched linear descriptor systems in discrete-time domain. Since the existing results for 
stability of switched state space systems suggest that the common Lyapunov functions 
condition should be less conservative than the commutation condition, we establish our 
approach based on common quadratic Lyapunov functions incorporated with linear matrix 
inequalities (LMIs). We show that if there is a common quadratic Lyapunov function for 
stability of all descriptor subsystems, then the switched system is stable under impulse-free 
arbitrary switching. This is a reasonable extension of the results in [17, 18], in the sense that if 
all descriptor subsystems are stable, and furthermore the descriptor matrix and all subsystem 
matrices are commutative pairwise, then there exists a common quadratic Lyapunov function 
for all subsystems, and thus the switched system is stable under impulse-free arbitrary 
switching. Furthermore, we show that if there is a common quadratic Lyapunov function 
for stability and certain £2 gain of all descriptor subsystems, then the switched system is 
stable and has the same £2 gain under impulse-free arbitrary switching. Since the results are 
consistent with those for switched state space systems when the descriptor matrix shrinks to 
an identity matrix, the results are natural but important extensions of the existing results. 
The rest of this article is organized as follows. Section 2 gives some preliminaries 
on discrete-time descriptor systems, and then Section 3 formulates the problem under 
consideration. Section 4 states and proves the stability condition for the switched linear 
discrete-time descriptor systems under impulse-free arbitrary switching. The condition 
requires in fact a common quadratic Lyapunov function for stability of all the subsystems, 
and includes the existing commutation condition [17, 18] as a special case. Section 5 extends 
the results to £2 gain analysis of the switched system under impulse-free arbitrary switching, 
and the condition to achieve the same stability and £2 gain properties requires a common 
quadratic Lyapunov function for all the subsystems. Finally, Section 6 concludes the article. 

2. Preliminaries 

Let us first give some preliminaries on linear discrete-time descriptor systems. Consider the 
descriptor system 
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Ex(k + 1) = Ax(k) + Bw(k) 

(2.1) 
z(k) = Cx(k), 

where the nonnegative integer k denotes the discrete time, x(k) £ lZ n is the descriptor 
variable, w{k) G TV is the disturbance input, z(k) £ V3 is the controlled output, £ £ lZ nxn , 
A g lZ n x ", B £ 72." x P and C £ "R.1 x " are constant matrices. The matrix £ may be singular and 
we denote its rank by r = rank E < n. 

Definition 1: Consider the linear descriptor system (2.1) with w = 0. The system has a unique 
solution for any initial condition and is called regular, if \zE — A\ ^ 0. The finite eigenvalues 
of the matrix pair (E,A), that is, the solutions of \zE — A\ =0, and the corresponding 
(generalized) eigenvectors define exponential modes of the system. If the finite eigenvalues lie 
in the open unit disc of z, the solution decays exponentially. The infinite eigenvalues of (E, A) 
with the eigenvectors satisfying the relations Ex\ = determine static modes. The infinite 
eigenvalues of (£, A) with generalized eigenvectors X/ c satisfying the relations Ex\ = and 
£x; c = x/t-i (k > 2) create impulsive modes. The system has no impulsive mode if and only if 
rank £ = deg |s£ — A\ (deg |z£ — A\). The system is said to be stable if it is regular and has 
only decaying exponential modes and static modes (without impulsive modes). I 

Lemma 1 (Weiertrass Form)[l, 2] If the descriptor system (2.1) is regular, then there exist two 
nonsingular matrices M and N such that 



MEN 



/ 



, MAN 



A 
l n -d 



(2.2) 



where d = deg |z£ — A\, J is composed of Jordan blocks for the finite eigenvalues. If the 
system (2.1) is regular and there is no impulsive mode, then (2.2) holds with d = r and / = 0. 
If the system (2.1) is stable, then (2.2) holds with d = r, ] = and furthermore A is Schur 
stable. I 

Let the singular value decomposition (SVD) of £ be 



L2 



£ n 




V T , £ u = diag{ui, • ■ • ,a r } 



(2.3) 



where c,'s are the singular values, U and V are orthonormal matrices (U T U = V T V = I). 
With the definitions 

'A u A X1 



V x = 



, U T AV 



A 2 \ A 2 2 
the difference equation in (2.1) (with w = 0) takes the form of 

Eu*i(k + 1) = A nXl (k) + A u s 2 (k) 
= A 2l x x (k) + A 2 2X 2 {k) . 



(2.4) 



(2.5) 



It is easy to obtain from the above that the descriptor system is regular and has not impulsive 
modes if and only if A 22 is nonsingular. Moreover, the system is stable if and only if A 22 is 
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nonsingular and furthermore EZf I An — ^12^22 2 1 ) * s Schur stable. This discussion will 

be used again in the next sections. 

Definition 2: Given a positive scalar 7, if the linear descriptor system (2.1) is stable and satisfies 

£ z T (j)z(j) < 0(x(O)) + 7 2 E w T {j)w{j) (2.6) 

1=0 j=o 

for any integer k > and any /2-bounded disturbance input w, with some nonnegative definite 
function </>(■), then the descriptor system is said to be stable and have £2 gain l ess than 7. I 
The above definition is a general one for nonlinear systems, and will be used later for switched 
descriptor systems. 

3. Problem formulation 

In this article, we consider the switched system composed of TV" linear discrete-time descriptor 

subsystems described by 

Ex(k + 1) = Ajx(k) + B { w(k) 

(3.1) 
z(k) = Cjx{k) , 

where the vectors x, w, z and the descriptor matrix £ are the same as in (2.1), the index i 
denotes the 2-th subsystem and takes value in the discrete set X = {1, 2, • • ■ ,M}, and thus the 
matrices Aj, B,-, Q together with £ represent the dynamics of the i-th subsystem. 
For the above switched system, we consider the stability and £2 gain properties under the 
assumption that all subsystems in (3.1) are stable and have £2 gain l ess than 7. As in the case 
of stability analysis for switched linear systems in state space representation, such an analysis 
problem is well posed (or practical) since a switched descriptor system can be unstable even if 
all the descriptor subsystems are stable and there is no variable (state) jump at the switching 
instants. Additionally, switchings between two subsystems can even result in impulse signals, 
even if the subsystems do not have impulsive modes themselves. This happens when the 
variable vector x(k r ), where k r is a switching instant, does not satisfy the algebraic equation 
required in the subsequent subsystem. In order to exclude this possibility, Ref. [19] proposed 
an additional condition involving consistency projectors. Here, as in most of the literature, 
we assume for simplicity that there is no impulse occurring with the variable (state) vector at 
every switching instant, and call such kind of switching impulse-free. 

Definition 3: Given a switching sequence, the switched system (3.1) with w = is said to 
be stable if starting from any initial value the system's trajectories converge to the origin 
exponentially, and the switched system is said to have £2 gain less than 7 if the condition 
(2.6) is satisfied for any integer k > 0. I 

In the end of this section, we state two analysis problems, which will be dealt with in Section 
4 and 5, respectively. 

Stability Analysis Problem: Assume that all the descriptor subsystems in (3.1) are stable. 
Establish the condition under which the switched system is stable under impulse-free 
arbitrary switching. 

£2 Gain Analysis Problem: Assume that all the descriptor subsystems in (3.1) are stable and 
have £2 gain less than 7. Establish the condition under which the switched system is also 
stable and has £2 gain less than 7 under impulse-free arbitrary switching. 
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Remark 1: There is a tacit assumption in the switched system (3.1) that the descriptor matrix 
£ is the same in all the subsystems. Theoretically, this assumption is restrictive at present. 
However, as also discussed in [17, 18], the above problem settings and the results later can 
be applied to switching control problems for linear descriptor systems. This is the main 
motivation that we consider the same descriptor matrix E in the switched system. For 
example, if for a single descriptor system Ex (k + 1) = Ax(k) + Bu(k) where u(k) is the control 
input, we have designed two stabilizing descriptor variable feedbacks u = K\X, u = K2X, and 
furthermore the switched system composed of the descriptor subsystems characterized by 
(£, A + BKi) and (E, A + BK2) are stable (and have £2 g am less than 7) under impulse-free 
arbitrary switching, then we can switch arbitrarily between the two controllers and thus can 
consider higher control specifications. This kind of requirement is very important when we 
want more flexibility for multiple control specifications in real applications. I 

4. Stability analysis 

In this section, we first state and prove the common quadratic Lyapunov function (CQLF) 
based stability condition for the switched descriptor system (3.1) (with w = 0), and then 
discuss the relation with the existing commutation condition. 



4.1 CQLF based stability condition 

Theorem 1: The switched system (3.1) (with w = 0) is stable under impulse-free arbitrary 
switching if there are nonsingular symmetric matrices P, G 1Z" X " satisfying for Vi G T that 



E T P t E > 
AjPfA, - E T PiE < 



(4.1) 
(4.2) 



and furthermore 



E T PiE = E T PjE, Mi,] el, i^ j. 



(4.3) 



Proof: The necessary condition for stability under arbitrary switching is that each subsystem 
should be stable. This is guaranteed by the two matrix inequalities (4.1) and (4.2) [20]. 
Since the rank of E is r, we first find nonsingular matrices M and N such that 



MEN = 


IrO 








Then, we obtain from (4.1) that 


(N T E T M T )(M- T P l M~ 1 ){MEN) = 


%1° 




where 

M' T P,M^ = 


' Hi 

.(P'n) 


P' 

T pi 
I 22 





> 0, 



(4.4) 



(4.5) 



(4.6) 



Since P,- (and thus M T PjM J ) is symmetric and nonsingular, we obtain PL > 
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Again, we obtain from (4.3) that 

(N T E T M T ){M- T P,M- 1 )(MEN) = (N T E T M T ){M- T P j M- 1 ){MEN) , (4.7) 

and thus 

~PL ol \pL n" 

(4.8) 



^11 




P> n Q 




which leads to Pjj = P\y Vi,j S I. From now on, we let PL = P\\ for notation simplicity. 
Next, let 



A' u A\ 2 

A' A~i 
n l\ ^22 



MAjN 

and substitute it into the equivalent inequality of (4.2) as 

(N T AjM T )(M- T P i M~ 1 )(MA i N) - (N T E T M T )(M' T P,M^)(MEN) < 

to reach 

An An 

<0, 



A' A 22 



(4.9) 

(4.10) 
(4.11) 



where 



An = (A' n ) T P n A l n - P u + (A 21) v , 12 



<P[ 2 ) A' n + {A' n ) P{ 2 A 2l + (A 21 ) P' 22 A 2i 
A12 = {A i ll ) T P ll A' l2 + (A^fPlzAin + (A 21 ) T (P[ 2 ) T A\ 2 + {A 2l ) T P l 22 A 22 
A 22 = (A[ 2 ) T P U A\ 2 + (A 22 ) T (P[ 2 ) T A[ 2 + (A[ 2 ) T P{ 2 A 22 + {A 22 ) T P l 22 A 22 . 



(4.12) 



At this point, we declare A' 22 is nonsingular from A 22 < 0. Otherwise, there is a nonzero 
vector v such that A 22 y = 0. Then, u T A 22 y < 0. However, by simple calculation, 



v T A 12 v = v T (A[ 2 ) T P u A[ 2 v > 
since Pu is positive definite. This results in a contradiction. 
Multiplying the left side of (4.11) by the nonsingular matrix 
right side by its transpose, we obtain 

{A\ 1 yp lx A\ l -p n * 



I -(A' 21 ) T (A 22 y T 
I 



(*) ] 



A 22 



<0, 



(4.13) 



and the 



(4.14) 



where A' n = A' n - A[ 2 (A' 22 ) l A\ v 

With the same nonsingular transformation x{k) = N~ l x(k) = [xj(k) xJ(k)] T , X\{k) £ 7J. r , all 

the descriptor subsystems in (3.1) take the form of 



xi{k + l) = A' u xi(k) + A[ 2 x 2 {k) 
= A 21 xi(fc) + A 22 .f 2 (fc), 



(4.15) 
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which is equivalent to 

x 1 (k + l)=A[ 1 x 1 (k) 

wiihx 2 (k) = -(A i 21 )- 1 A i 11 Xi{k). It is seen from (4.14) that 



(4.16) 



(A 1 u yp u A\ 1 -p 11 <o, 



(4.17) 



which means that all A'^'s are Schur stable, and a common positive definite matrix P\\ exists 
for stability of all the subsystems in (4.16). Therefore, X\(k) converges to zero exponentially 
under impulse-free arbitrary switching. The X2CO P ar t is dominated by X\(k) and thus also 
converges to zero exponentially. This completes the proof. I 

Remark 2: When E = I and all the subsystems are Schur stable, the condition of Theorem 
1 actually requires a common positive definite matrix P satisfying AjPAj — P < for Vz £ 
I, which is exactly the existing stability condition for switched linear systems composed of 
x(k + 1) = Ajx(k) under arbitrary switching [12]. Thus, Theorem 1 is an extension of the 
existing result for switched linear state space subsystems in discrete-time domain. I 

Remark 3: It can be seen from the proof of Theorem 1 that xTPiiX-y is a common quadratic 
Lyapunov function for all the subsystems (4.16). Since the exponential convergence of X\ 
results in that of Xi, we can regard x^P\\X\ as a common quadratic Lyapunov function for the 
whole switched system. In fact, this is rationalized by the following equation. 

x T E T PjEx = {N- 1 x) T {MEN) T {M- T P i M- 1 ){MEN)(N- 1 x) 
-1 T 



1,0 





'11 



p' 

J 12 



(pi \T pi 
^121 ^22 





1,0 






x\ 

*2 



x\P xl x x 



(4.18) 



Therefore, although E T PjE is not positive definite and neither is V(x) = x T E T PjEx, we can 
regard this V(x) as a common quadratic Lyapunov function for all the descriptor subsystems 
in discrete-time domain. I 

Remark 4: The LMI conditions (4.1)-(4.3) include a nonstrict matrix inequality, which may not 
be easy to solve using the existing LMI Control Toolbox in Matlab. As a matter of fact, the 
proof of Theorem 1 suggested an alternative method for solving it in the framework of strict 
LMIs: (a) decompose £ as in (4.4) using nonsingular matrices M and N; (b) compute MAjN 
for Vi G X as in (4.9); (c) solve the strict LMIs (4.11) for V7 £ I simultaneously with respect to 

Pu pI ' 

?H > 0, P{ 2 and P' 21 ; (d) compute the original P t with P, = M 1 



12 
(^12) ^22 



M. 



Although we assumed in the above that the descriptor matrix is the same for all the 
subsystems (as mentioned in Remark 1), it can be seen from the proof of Theorem 1 that what 
we really need is the equation (4.4). Therefore, Theorem 1 can be extended to the case where 
the subsystem descriptor matrices are different as in the following corollary. 
Corollary 1: Consider the switched system composed of M linear descriptor subsystems 



Ejx{k + l) = Ajx(k), 



(4.19) 
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where E, is the descriptor matrix of the ith subsystem and all the other notations are the same 
as before. Assume that all the descriptor matrices have the same rank r and there are common 
nonsingular matrices M and N such that 



ME,N 



1,0 





, VieZ. 



(4.20) 



Then, the switched system (4.19) is stable under impulse-free arbitrary switching if there are 
symmetric nonsingular matrices P, £ lZ nxn (/ = !,■■■ ,Af) satisfying for V7 £ T 



and furthermore 



EjPjE, > 0, AjPiA, - EfPiEi < 



EjPiEj = EJPjEj, Vt,; el, i^j. 



(4.21) 

(4.22) 

I 



4.2 Relation with existing commutation condition 

In this subsection, we consider the relation of Theorem 1 with the existing commutation 
condition proposed in [17]. 

Lemma 2:([17]) If all the descriptor subsystems are stable, and furthermore the matrices E, 
A\, ■ • • , Aj^f are commutative pairwise, then the switched system is stable under impulse-free 
arbitrary switching. I 

The above lemma establishes another sufficient condition for stability of switched linear 
descriptor systems in the name of pairwise commutation. It is well known [12] that in the 
case of switched linear systems composed of the state space subsystems 



x(k + l) = Ajx(k), iel, 



(4.23) 



where all subsystems are Schur stable and the subsystem matrices commute pairwise (AjA; 
AjAj, \/i,j 6 T), there exists a common positive definite matrix P satisfying 



A\ PAi-P < . 



(4.24) 



One then tends to expect that if the commutation condition of Lemma 2 holds, then a common 

quadratic Lyapunov function V(x) = x E PjEx should exist satisfying the condition of 

Theorem 1. This is exactly established in the following theorem. 

Theorem 2: If all the descriptor subsystems in (3.1) are stable, and furthermore the matrices 

£, A\, ■ ■ ■ , Apj are commutative pairwise, then there are nonsingular symmetric matrices P,'s 

(i = 1, ■ ■ ■ , Af) satisfying (4.1)-(4.3), and thus the switched system is stable under impulse-free 

arbitrary switching. 

Proof: For notation simplicity, we only prove the case of A^ = 2. Since (E,A\) is stable, 

according to Lemma 1, there exist two nonsingular matrices M, N such that 



MEN 



1,0 




AL4jN : 



Al 
I„-r 



(4.25) 
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where A\ is a Schur stable matrix. Here, without causing confusion, we use the same notations 
M, N as before. Defining 

r Wi \N 2 

w 3 w 4 

and substituting it into the commutation condition EAj = A\E with 



N~ r M-' 1 



(4.26) 



-l»>r-ll 



{ME^iN-^-^iMA^) = (MA 1 N)(N" 1 M" 1 )(M£JV) 



(4.27) 



we obtain 



WjAj W 2 




AjWj 
W 3 



(4.28) 



Thus, WiAi = AiWi, W 2 = 0, W 3 = 0. 

Now, we use the same nonsingular matrices M, N for the transformation of A 2 and write 



MA 2 N 



A 2 X X 
X 2 X 



According to another commutation condition EA 2 = A 2 E, 



W 1 A 2 WjXj 




A 2 Wj 
X 2 W! 



(4.29) 



(4.30) 



holds, and thus WiA 2 = A 2 Wj, W\X\ = 0, X 2 Wi = 0. Since NM is nonsingular and W 2 = 
0, W3 = 0, Vi\ has to be nonsingular. We obtain then Xj = 0, X 2 = 0. Furthermore, since 
(£, A 2 ) is stable, A 2 is Schur stable and X has to be nonsingular. 
The third commutation condition A\ A 2 = A 2 Aj results in 



A!W"iA 2 
W 4 X 



A 2 W!Ai 
XW 4 



(4.31) 



We have A- l W 1 A 2 = A 2 W 1 A 1 . Combining with W 1 A 1 = A^Wi, W\A 2 = A2W1, we obtain 
that 

WjA^ = A!W!A 2 = A 2 W!A! = W!A 2 A! (4.32) 

which implies Aj and A 2 are commutative (A^A 2 = A 2 Aj) since Wj is nonsingular. 
To summarize the above discussion, we get to 



MA 2 N 



A 2 
X 



(4.33) 



where A 2 is Schur stable, X is nonsingular, and Aj A 2 = A 2 Aj . According to the existing result 
[12], there is a common positive definite matrix P\\ satisfying AjPnAj — P\\ < 0, i = 1,2. 
Then, with the definition 

Pu 



P 2 = M 1 



-I 



M, 



(4.34) 
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it is easy to confirm that 



{MEN) T (M~ T P l M~ 1 )(MEN) 







> 



(4.35) 



and 



<0, 



(MA 1 N) T {M- T P 1 M- 1 ){MA 1 N) - (MENfiM^P^-^iMEN) 
'AfPuAj-Pu " 
-I 

(MA 2 N) T (M- T P 2 M- 1 )(MA 2 N) - (MEN) T {M- T P 2 M- 1 ){MEN) 
'AjP u A 2 -Pii 

-X T X 



(4.36) 



< 0. 



Since P\\ is common for i = 1, 2 and N is nonsingular, (4.35) and (4.36) imply that the matrices 
in (4.34) satisfy the conditions (4.1)-(4.3). I 

It is observed from (4.34) that when the conditions of Theorem 2 hold, we can further choose 
Pi = P 2 , which certainly satisfies (4.3). Since the actual Lyapunov function for the stable 
descriptor system Ex[k + 1] = A,x[fc] takes the form of V(x) = x T E T PiEx (as mentioned 
in Remark 3), the commutation condition is more conservative than the LMI condition in 
Theorem 1. However, we state for integrity the above observation as a corollary of Theorem 
2. 

Corollary 2: If all the descriptor subsystems in (3.1) are stable, and furthermore the matrices 
E, A\, • • • , Apj are commutative pairwise, then there is a nonsingular symmetric matrix P 
satisfying 



E T PE > 
AjPA,-E T PE < 0, 

and thus the switched system is stable under impulse-free arbitrary switching. 



(4.37) 
(4.38) 



5. £2 9ain analysis 

In this section, we extend the discussion of stability to C 2 gain analysis fro the switched linear 
descriptor system under consideration. 

Theorem 3: The switched system (3.1) is stable and the C 2 gain is less than 7 under impulse-free 
arbitrary switching if there are nonsingular symmetric matrices P; G 7J nx " satisfying for Mi G 
I that 



E T P { E > 

AfPjA, - E T PiE + CfC, AjP,Bi 

BfP t A, BfP,B t - j 2 I 



<0 



(5.1) 
(5.2) 



together with (4.3). 
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Proof: Since (5.1) is the same as (4.1) and (5.2) includes (4.2), we conclude from Theorem 1 that 
the switched descriptor system is exponentially stable under impulse-free arbitrary switching. 
What remains is to prove the £2 g am property. 

Consider the Lyapunov function candidate V(x) = x E P{Ex, which is always nonnegative 
due to (5.1) and always continuous due to (4.3). Then, on any discrete-time interval where the 
z'-th subsystem is activated, the difference of V(x) along the system's trajectories satisfies 

V(x(k + 1)) - V(x(k)) = x T {k + l)E T P,Ex{k + 1) - x T {k)E T PjEx{k) 

= {Ex{k + l)) T Pj{Ex{k + 1)) - x T {k)E T PjEx{k) 

= (Ajx{k) + B,iv{k)) T P^A.xik) + B t w(k)) - x T {k)E T Pj Ex(k) 



" *(*) " 


T r 


H>(fc) 




"*(*)" 


T r 


w(k) 





AjP t A, - E 


r P,E Aj 


PiBi' 




rx(k)i 


BfPiAi BjPfr _ 




w(k) 


-cjc { " 




\x(k)~ 




y 2 I 




w{k) 









-z 1 (fc)z(fc) + Yw l (k)w(k) , 



(5.3) 



where the condition (5.2) was used in the inequality. 

Now, for an impulse-free arbitrary piecewise constant switching signal and any given k > 0, 
suppose k\ < ki < ■ ■ ■ < k r (r > 1) be the switching points of the switching signal on the 
discrete-time interval [0,k). Then, according to (5.3), we obtain 

v(x(k + i))-v(x(k+)) < £ {-z t (/>(/) + tV (/>(;)} 

V(x(KT))-V(x(tf_ 1 ))< £ {-z T (j)z(j) + 7 2 w T (j)iv(j)} 



where 



V(x(k^))-V(x(k+))< £ {-z r (;)z(/) + 7 V(;>(/)} 
j=ki 

V(x(k-))-V(x(0)) < £ {-z r (;)z(/) + 7V(;>(/)}, 
1=0 



V(x(k+)) = lim V(x(k)), V(x(kJ)) = lim V(x(k)) 



k^kj-0 



(5.4) 



(5.5) 



However, due to the condition (4.3), we obtain V(x(k3~)) = V(x(k: )) at all switching instants. 
Therefore, summing up all the inequalities of (5.4) results in 



V(x(k + 1))-V(x(0)) <£{-z T (j)z(j) + 7 2 zv T (j)w(j)} 
1=0 



(5.6) 
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Since V(x(k + 1)) > 0, we obtain that 

£> T (/)z(/) < ^(0)) + 7 2 E^ T (7>(;)/ (5-7) 

7=0 7=0 

which implies the £2 g a i n of the switched system is less than 7. I 

Remark 5: When E = I, the conditions (5.1)-(5.2) and (4.3) require a common positive definite 
matrix P satisfying 



AjP.A, - P { + Cjd AjPiB, 

BfP.Ai BfPiB, - 7 2 i 



< (5.! 



for all Vi £ T, which is the same as in [15]. Thus, Theorem 3 extended the £2 gain analysis 
result from switched time space systems to switched descriptor systems in discrete-time 
domain. In addition, it can be seen from the proof that V(x) = x E PjEx plays the important 
role of a common quadratic Lyapunov function for stability and £2 gain 7 of all the descriptor 
subsystems. I 

6. Concluding remarks 

We have established a unified approach to stabilility and £2 gain analysis for switched linear 
discrete-time descriptor systems under impulse-free arbitrary switching. More precisely, 
we have shown that if there is a common quadratic Lyapunov function for stability of 
all subsystems, then the switched system is stable under impulse-free arbitrary switching. 
Furthermore, we have extended the results to £2 gain analysis of the switched descriptor 
systems, also in the name of common quadratic Lyapunov function approach. As also 
mentioned in the remarks, the common quadratic Lyapunov functions proposed are not 
positive definite with respect to all states, but they actually play the role of a Lyapunov 
function as in classical Lyapunov stability theory. The approach in this article is unified in 
the sense that it is valid for both continuous-time [21] and discrete-time systems, and it takes 
almost the same form in both stability and £2 gain analysis. 
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1. Introduction 

Switched systems are a class of hybrid systems consisting of several subsystems (modes of 
operation) and a switching rule indicating the active subsystem at each instant of time. In 
recent years, considerable efforts have been devoted to the study of switched system. The 
motivation of study comes from theoretical interest as well as practical applications. 
Switched systems have numerous applications in control of mechanical systems, the 
automotive industry, aircraft and air traffic control, switching power converters, and many 
other fields. The basic problems in stability and design of switched systems were given by 
(Liberzon & Morse, 1999). For recent progress and perspectives in the field of switched 
systems, see the survey papers (DeCarlo et al., 2000; Sun & Ge, 2005). 

The stability analysis and stabilization of switching systems have been studied by a number 
of researchers (Branicky, 1998; Zhai et al., 1998; Margaliot & Liberzon, 2006; Akar et al., 
2006). Feedback stabilization strategies for switched systems may be broadly classified into 
two categories in (DeCarlo et al., 2000). One problem is to design appropriate feedback 
control laws to make the closed-loop systems stable for any switching signal given in an 
admissible set. If the switching signal is a design variable, then the problem of stabilization 
is to design both switching rules and feedback control laws to stabilize the switched 
systems. For the first problem, there exist many results. In (Daafouz et al., 2002), the switched 
Lyapunov function method and LMI based conditions were developed for stability analysis 
and feedback control design of switched linear systems under arbitrary switching signal. 
There are some extensions of (Daafouz et al., 2002) for different control problem (Xie et al., 
2003; Ji et al., 2003). The pole assignment method was used to develop an observer-based 
controller to stabilizing the switched system with infinite switching times (Li et al., 2003). 
It is should be noted that there are relatively little study on the second problem, especially 
for uncertain switched systems. Ji had considered the robust Hco control and quadratic 
stabilization of uncertain discrete-time switched linear systems via designing feedback 
control law and constructing switching rule based on common Lyapunov function approach 
(Ji et al., 2005). The similar results were given for the robust guaranteed cost control problem 
of uncertain discrete-time switched linear systems (Zhang & Duan, 2007). Based on multiple 
Lyapunov functions approach, the robust Hco control problem of uncertain continuous-time 
switched linear systems via designing switching rule and state feedback was studied (Ji et 
al., 2004). Compared with the switching rule based on common Lyapunov function 
approach (Ji et al., 2005; Zhang & Duan, 2007), the one based on multiple Lyapunov 
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functions approach (Ji et al., 2004) is much simpler and more practical, but discrete-time case 
was not considered. 

Motivated by the study in (Ji et al., 2005; Zhang & Duan, 2007; Ji et al., 2004), based on the 
multiple Lyapunov functions approach, the robust control for a class of discrete-time 
switched systems with norm-bounded time-varying uncertainties in both the state matrices 
and input matrices is investigated. It is shown that a state-depended switching rule and 
switched state feedback controller can be designed to stabilize the uncertain switched linear 
systems if a matrix inequality based condition is feasible and this condition can be dealt 
with as linear matrix inequalities (LMIs) if the associated scalar parameters are selected in 
advance. Furthermore, the parameterized representation of state feedback controller and 
constructing method of switching rule are presented. All the results can be considered as 
extensions of the existing results for both switched and non-switched systems. 

2. Problem formulation 

Firstly, we consider a class of uncertain discrete-time switched linear systems described by 

lx(k + l) = (A a{k) + AA a{k) ) x(k) + (B a{k) +AB r7{k) ) u(k) 

K^) S^ (1) 

y( k )= c a(kf( k ) 

where x(k) e R" is the state, u(k) e R m is the control input, y(k) s R ? is the measurement 
output and a(k) e S = {1,2,---N} is a discrete switching signal to be designed. Moreover, 
a(k) = i means that the ith subsystem {A^B^C^ is activated at time k (For notational 
simplicity, we may not explicitly mention the time-dependence of the switching signal 
below, that is, a(k) will be denoted as a in some cases). Here A, , B { and C, are constant 
matrices of compatible dimensions which describe the nominal subsystems. The uncertain 
matrices AA, and AB ; are time-varying and are assumed to be of the forms as follows. 

AA, = M m F m (k)N a , AB, = M b ,F b ,(k)N b , (2) 

where M ai , N aj , M bj , N hi are given constant matrices of compatible dimensions which 
characterize the structures of the uncertainties, and the time-varying matrices F ai (k) and 
F bi (k) satisfy 

#(*)U*) *',#(*)*£(*)* * v ' eE (3) 

where I is an identity matrix. 

We assume that no subsystem can be stabilized individually (otherwise the switching 

problem will be trivial by always choosing the stabilized subsystem as the active 

subsystem). The problem being addressed can be formulated as follows: 

For the uncertain switched linear systems (1), we aim to design the switched state feedback 

controller 

u(k)=K a x(k) (4) 

and the state-depended switching rule a(x(k)) to guarantee the corresponding closed-loop 
switched system 
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x(k + l) = [A a+ AA a+ (B a + AB a )KJx(k) 



(5) 



is asymptotically stable for all admissible uncertainties under the constructed switching 
rule. 

3. Main results 

In order to derive the main result, we give the two main lemmas as follows. 

Lemma 1: (Boyd, 1994) Given any constant e and any matrices M,N with compatible 

dimensions, then the matrix inequality 

MFN + N T F T M T < sMM T + s^N T N 

holds for the arbitrary norm-bounded time- varying uncertainty F satisfying F F <I . 
Lemma 2: (Boyd, 1994) (Schur complement lemma) Let S,P,Q be given matrices such that 
Q = Q T / P = P T / then 

<0oQ<0,P-S r Q 4 S<0. 

s Q\ 

A sufficient condition for existence of such controller and switching rule is given by the 
following theorem. 

Theorem 1: The closed-loop system (5) is asymptotically stable when AA, = AB, = if there 
exist symmetric positive definite matrices X, e R" x " , matrices G, e R" x " , Y, e R mx " , scalars 
£j > (i e H) and scalars /L < (i,j e S,/^, = -1) such that 



"IV G < T + 


G,- 


x,) 


* 


* 


* 




* 


jeE 
















Afii + 


w 




-x, 


* 


* 




* 


G, 









x n x a 


* 




* 


G, 












A i2 X 2 




* 
* 


G, 


















^-iN-^N _ 



< Vz e S 



(6) 



is satisfied ( * denotes the corresponding transposed block matrix due to symmetry), then 
the state feedback gain matrices can be given by (4) with 



K,=r,G,- 



and the corresponding switching rule is given by 

a(x(k)) = argmin{x T (fc)Xr 1 x(fc)} 



(7) 



(8) 



Proof . Assume that there exist G i ,X i ,Y i ,s i and /L such that inequality (6) is satisfied. 
By the symmetric positive definiteness of matrices X, , we get 
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which is equal to 



GjxfG^G] + G,-X, 



It follows from (6) and /L < that 



Y^GjxfG, * * 



O, 



<0 



(9) 



where r, = [G,,G„. ■ -G,] 1 , O, = diag{l/ ^X v l / ^ 2 X 2 ,- ■ -,1 / 1,^X^,1 / \ i+1) X i+ll - ■ -,1 / A lN X N } 

Pre- and post- multiplying both sides of inequality (9) by diagjG," 1 ,!,!} 1 and diagfG, 71 ,!,!} , 
we get 



E^: 1 * * 

A, + B;iCj -X, * 

n, o o. 



<o 



(10) 



where n, ■= [1,1,- -if . 

By virtue of the properties of the Schur complement lemma, inequality (10) is equal to 



'-x; 1 + s v x < rl 


-x?) 
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jeS./Vi 






A +B t K t 




~ X i\ 



<0 



Letting P { = X, and applying Schur complement lemma again yields 
(A + B^fP^A, + B,K,) -P i + £ V P < - P /> < ° 

;sE,j*i 

Since P ; = X,~ (Vj e E) , the switching rule (8) can be rewritten as 

a-(x(k)) = argmin{x T (k)P 1 x(k)}. 
By (13), <j(k) = i implies that 

x T {k){P l - Pj)x{k) < 0, V; e E,; * j. 
Multiply the above inequalities by negative scalars /L for each j e E, j ^ i and sum to get 



(11) 



r w I v p <- p /> 



x(fc)>0 



(12) 

(13) 
(14) 

(15) 
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Associated with the switching rule (13), we take the multiple Lyapunov functions V(x(k)) 
as 

V„ {k) (x(k)) = x T (k)P„ {k) x(k) (16) 

then the difference of V(x(k)) along the solution of the closed-loop switched system (5) is 

A V = V(x(k + 1)) - V(x(k)) = x J (k + l)P a{k+1) x(k + 1) - x T (k)P a(k) x(k) 

At non-switching instant, without loss of generality, letting a(k + l) = cr{k) = i(i e E) , and 
applying switching rule (13) and inequality (15), we get 

AV =x T (k + l)P,x(k + 1) - x J (k)P,x(k) = x T (/c)[(A, + Bfcf P,(A, + B,K ( ) - P,~\x(k) < (17) 

It follows from (12) and (15) that AV < holds. 

At switching instant, without loss of generality, let a(k + 1) = j,<y(k) = i(i,j eS,i* j) to get 

AV = x T (k +l)P jX (k + 1) - x T (k)P t x(k) < x T (k + l)P,x(k +1) - x T (k)P,x(k) <0 (18) 

It follows from (17) and (18) that AV < holds. In virtue of multiple Lyapunov functions 

technique (Branicky, 1998), the closed-loop system (5) is asymptotically. This concludes the 

proof. 

Remark 1: If the scalars /L are selected in advance, the matrices inequalities (19) can be 

converted into LMIs with respect to other unknown matrices variables, which can be 

checked with efficient and reliable numerical algorithms available. 

Theorem 2: The closed-loop system (5) is asymptotically stable for all admissible 

uncertainties if there exist symmetric positive definite matrices X, e R" x " , matrices 



G,.£] 



, Yi-el 



' , scalars e t > (i e S) and scalars /L < (i,j e H,/l n = -1) such that 



l4y(G, T +G,-X,) 



& + W 


0, 


* 


* 


* 


* 


N al G, 





- £i I 


* 


* 


* 
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A l2 X 2 



is satisfied, where 







An X nJ 



< Vz e E 



(19) 



e, 



-X l+£l [M m Ml+M bl Ml], 



then the state feedback gain matrices can be given by (4) with 



K,=Y,G,- 



(20) 
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and the corresponding switching rule is given by 



a(x(k)) = argmin{x T (fc)XrV(fc)} 



(21) 



Proof. By theorem 1, the closed-loop system (5) is asymptotically stable for all admissible 
uncertainties if that there exist G i ,X i ,Y i and /L such that 



E^fGj+GrXi) 



Afii+W ®i * 

r, o <d, 



<0 



where T, =[G,,G,,---G,] , 

<D, = diagjl / Z n X lr l I X i2 X 2 ,~; 1 / Vifr-1 / Ww •"' 1 / A^n! ' 



which can be rewritten as 



A, + M;F;(fc)N; + nJP?(1c)mJ < 



where 



XV G < T+G <- X <) 



,'fc- 



r, o <d, 



F l (k) = dmg(F m (k),F u (k)), fj. 



It follows from Lemma 1 and F?(t)F?(t) <I that 



a,+m,mJ +nJn, <o 
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(22) 



(23) 



By virtue of the properties of the Schur complement lemma, inequality (19) can be rewritten 



Xv G ? +G <- x ') 



A i G i+ B i Y i 



N m G t 






N hi Y, 



(S) * * * 

(D ! * 

-s,.I * 

-ftl 



<0 VieH 



(24) 



It is obvious that inequality (24)is equal to inequality (19), which finished the proof. 

Let the scalars /L = and X, = X = X , it is easily to obtain the condition for robust stability 

of the closed-loop system (5) under arbitrary switching as follows. 
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Corollary 1: The closed-loop system (5) is asymptotically stable for all admissible 
uncertainties under arbitrary switching if there exist a symmetric positive definite matrix 
X, e R" x " , matrices G, e R rax " , Y, e R mx " , scalars e, > and such that 



-G/-G.+X,. 



A l G l + B l Y l 



N bl Y, 
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o,. 
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* 





- £i I 


* 








-¥_ 



<0 VieH 



(25) 



is satisfied, where ; = -X, + s^M^M^ + M bi M bi ] , then the state feedback gain matrices can 
be given by (4) with 



K,=Y,G,- 



(26) 



4. Example 

Consider the uncertain discrete-time switched linear system (1) with N =2. The system 
matrices are given by 



"1.5 




1.5" 

-1.2 


A = 


)\m a = 


"0.5" 
0.2 


"1.2 

0.6 


" 
1.2 


,B 2 = 


_l} M - = 


"0.3" 
0.4 



N al =[0.4 0.2],M. 



bi 



N a2 =[0.3 0.2],M, 



ia 



0.3 
0.4 



0.3 
0.3 



,N n =[02], 



,N M =[0.1]. 



Obviously, the two subsystems are unstable, and it is easy to verify that neither subsystem 
can be individually stabilized via state feedback for all admissible uncertainties. Thus it is 
necessary to design both switching rule and feedback control laws to stabilize the uncertain 
switched system. Letting Xyi = -10 and A^ = -10 , the inequality (19) in Theorem 1 is 
converted into LMIs. Using the LMI control toolbox in MATLAB, we get 



X-, 



41.3398 -8 



70001 



,X 



-8.7000 86.6915J 



41.3415 -8.6656 

-8.7540 86.4219 

38.1665 -8.6003 

-8.6186 93.6219 





"38.1986 -8.6432" 


2 _ 


-8.6432 93.8897 


,*i = 


"-51.2846" 

-26.5670 


T 


h- 


"-44.3564" 
54.4478 


T 



^=56.6320,^=24.3598 
With K t = Y t G^ , the switched state feedback controllers are 



K-y =[-1.4841 -1.1505], K 2 =[-1.0527 0.4849]. 
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It is obvious that neither of the designed controllers stabilizes the associated subsystem. 
Letting that the initial state is x =[-3,2] and the time-varying uncertain 
F ia (k) = F ib (k) = f(k) (i = 1,2) as shown in Figure 1 is random number between -1 and 1, the 
simulation results as shown in Figure 2, 3 and 4 are obtained, which show that the given 
uncertain switched system is stabilized under the switched state feedback controller 
together with the designed switching rule. 
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Fig. 1. The time- varying uncertainty f(k) 
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Fig. 2. The state response of the closed-loop system 
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5. Conclusion 

This paper focused on the robust control of switched systems with norm-bounded 
time-varying uncertainties with the help of multiple Lyapunov functions approach and 
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matrix inequality technique. By the introduction of additional matrices, a new condition 
expressed in terms of matrices inequalities for the existence of a state-based switching 
strategy and state feedback control law is derived. If some scalars parameters are selected in 
advance, the conditions can be dealt with as LMIs for which there exists efficient numerical 
software available. All the results can be easily extended to other control problems 
(H 2 ,H„ control, etc.). 
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Fig. 3. The switching signal 
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Fig. 4. The state trajectory of the closed-loop system 
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1. Introduction 

Multitone modulations are today frequently used modulation techniques that enable 
optimum utilization of the frequency band provided on non-ideal transmission carrier 
channel (Bingham, 2000). These modulations are used with especially in data transmission 
systems in access networks of telephone exchanges in ADSL (asymmetric Digital Subscriber 
Lines) and VDSL (Very high-speed Digital Subscriber Lines) transmission technologies, in 
systems enabling transmission over power lines - PLC (Power Line Communication), in 
systems for digital audio broadcasting (DAB) and digital video broadcasting (DVB) [10]. 
And, last but not least, they are also used in WLAN (Wireless Local Area Network) 
networks according to IEEE 802.11a, IEEE 802.11g, as well as in the new WiMAX technology 
according to IEEE 802.16. This modulation technique makes use of the fact that when the 
transmission band is divided into a sufficient number of parallel subchannels, it is possible 
to regard the transmission function on these subchannels as constant. The more subchannels 
are used, the more the transmission function approximates ideal characteristics (Bingham, 
2000). It subsequently makes equalization in the receiver easier. However, increasing the 
number of subchannels also increases the delay and complication of the whole system. The 
dataflow carried by individual subchannels need not be the same and the number of bytes 
carried by one symbol in every subchannel is set such that it maintains a constant error rate 
with flat power spectral density across the frequency band used. The mechanism of 
allocating bits to the carriers is referred to as bit loading algorithm. The resulting bit-load to 
the carriers thus corresponds to an optimum distribution of carried information in the 
provided band at a minimum necessary transmitting power. 

In all the above mentioned systems the known and well described modulation DMT 
(Discrete MultiTone) (Bingham, 2000) or OFDM (Orthogonal Frequency Division 
Multiplexing) is used. As can be seen, the above technologies use a wide spectrum of 
transmission media, from metallic twisted pair in systems ADSL and VDSL, through radio 
channel in WLAN and WiMAX to power lines in PLC systems. 

Using multitone modulation, in this case DMT and OFDM modulations, with adaptive bit 
loading across the frequency band efficient data transmission is enabled on higher 
frequencies than for which the transmission medium was primarily designed (xDSL, PLC) 
and it is impossible therefore to warrant here its transfer characteristics. In terrestrial 
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transmission the relatively long symbol duration allows effective suppression of the 
influence of multi-path signal propagation (DAB, DVB, WiMAX, WLAN). 
Unfortunately, DMT and OFDM modulation ability will fail to enable quite an effective 
utilization of transmission channels with specially formed spectral characteristic with sharp 
transients, which is a consequence of individual subchannel frequency characteristic in the 
form of sine function. It is also the reason for the transmission rate loss on channels with the 
occurrence of narrow-band noise disturbance, both metallic and terrestrial. Moreover, multi- 
path signal propagation suppression on terrestrial channels is achieved only when the delay 
time is shorter than symbol duration. For these reasons the available transmission rate is 
considerably limited in these technologies. 

Alternative modulation techniques are therefore ever more often sought that would remove 
the above described inadequacies. The first to be mentioned was the DWMT modulation 
(Discrete Wavelet MultiTone) (Sandberg & Tzanes, 1995). This technique using the FWT 
(Fast Wavelet Transform) transform instead of the FFT transform in DMT or OFDM enabled 
by changing the carrier shape and reducing the sine function side lobes from - 13 dB to - 45 
dB a reduction of the influence of some of the above limitations. The main disadvantage of 
DWMT was the necessity to modulate carriers with the help of one-dimensional Pulse- 
Amplitude Modulation (PAM) instead of two-dimensional QAM, as with the DMT or 
OFDM system, i.e. a complex number implementing the QAM modulator bank. Another 
drawback was the high computational complexity. 

Another modulation method, which is today often mentioned, is the filter bank modulation, 
referred to as FMT (Filtered MultiTone or Filter bank MultiTone) (Cherubini et al.2000). 
FMT modulation represents a modulation technique using filter banks to divide the 
frequency spectrum. The system input is complex symbols, obtained with the help of QAM 
modulation, similar to classical DMT. The number of bits allocated to individual carriers is 
also determined during the transmission initialization according to the levels of interference 
and attenuation for the given channel, the same as with DMT. By upsampling the input 
signals their spectra will be periodized; subsequent filtering will select the part which will 
be transmitted on the given carrier. The filters in individual branches are frequency-shifted 
versions of the filter in the first branch, the so-called prototype filter - the lowpass filter 
(Cherubini et al.2000). Thanks to the separation of individual subchannel spectra, the 
interchannel interferences, ICI, are, contrary to the DMT, severely suppressed, down to a 
level comparable with the other noise. On the other hand, the intersymbol interferences, ISI, 
occur on every subchannel, event if the transmission channel is ideal (Benvenuto et al., 
2002). Therefore, it is necessary to perform an equalization of not only the transmission 
channel but also the filters. This equalization may be realized completely in the frequency 
domain. FMT also facilitates the application of frequency division duplex, because there is 
no power emission from one channel into another. 

2. DMT and OFDM modulations 

A signal transmitted by DMT or OFDM modulator can be described as shown by equation: 

x, N-l . , 2n '" C 



1 oo N-l 2nj_t 
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where 

h(t) { X f ° r fe <°' T ), /i= l = ^, r = 2N, Tcp= ^ 

W | otherwise T 2N f s f s 



and T„ 



2N + CP 



In DMT modulation, N - 1 is the number of carriers and so 2N is the number of samples in 
one symbol, k is the ordinal number of symbol, i is the carrier index, and X,* is the QAM 
symbol of i th carrier of k th symbol. In OFDM modulation all 2N carriers are modulated 
independently, and so the output signal x(f) is complex. The symbols are shaped by 
a rectangular window h(n), therefore the spectrum of each carrier is a sinc(f) function. The 
individual carriers are centred at frequencies fi and mutually overlapped. The transmission 
through the ideal channel enables a perfect demodulation of the DMT or OFDM signal on 
the grounds of the orthogonality between the individual carriers, which is provided by the 
FFT transformation. 

However, the transmission through non-ideal channels, mentioned in the first section, leads 
to the loss of orthogonality and to the occurrence of Inter-Symbol (ISI) and Inter-Carrier 
Interferences (ICI). To suppress the effect of the non-ideal channel, time intervals of duration 
Tcp (so-called cyclic prefixes) are inserted between individual blocks in the transmitted data 
flow in the transmitter. The cyclic prefix (CP) is generated by copying a certain number of 
samples from the end of next symbol. In the receiver the impulse response of the channel is 
reduced by digital filtering, called Time domain EQualizer (TEQ), so as not to exceed the 
length of this cyclic prefix. The cyclic prefix is then removed. This method of transmission 
channel equalisation in the DMT modulation is described in [6]. 

The spectrum of the carriers is a sincif) function and so the out-of-transmitted-band emission 
is much higher. There is a problem with the duplex transmission realisation by Frequency 
dividing multiplex (FDM) and with the transmission medium shared by another 
transmission technology. Figure 1 shows an example of ADSL technology. In ADSL2+ the 
frequency band from 7 th to 31 st carrier is used by the upstream channel and from 32 nd to 
511 th carriers by the downstream channel. The base band is used by the plain old telephone 
services (POTS). 

In ADSL, the problem with out-of-transmitted-band emission is solved by digital filtering of 
the signal transmitted using digital IIR filters. The out-of-transmitted-band emission is 
reduced (see Fig. 1.), but this filtering participates significantly in giving rise to ICI and ISI 
interferences. The carriers on the transmission band border are degraded in particular. 
Unfortunately, additional filtering increases the channel equalization complexity, because 
channel with transmit and receive filters creates a band-pass filter instead of low-pass filter. 
The difficulty is greater in upstream direction especially for narrow band reason. Therefore, 
a higher order of TEQ filter is used and the signal is sampled with two-times higher 
frequency compared to the sampling theorem. Also, when narrow-band interference 
appears in the transmission band, it is not only the carriers corresponding to this band that 
are disturbed but also a whole series of neighbouring carriers. 

The above disadvantages lead to a suboptimal utilization of the transmission band and to a 
reduced data rate. This is the main motivation for designing a new realisation of the MCM 
modulation scheme. Recently, a filter bank realisation of MCM has been the subject of 
discussion. This method is called Filtered MultiTone modulation (FMT). 
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upstream without filtering 
downstream without filtering 
upstream with filtering 
downstream with filtering 



tone [-] 



150 



Fig. 1. SNR comparison of upstream and part of downstream frequency bands of ADSL2+ 
technology with and without additional digital filtering. PSD= -40 dBm/Hz, AWGN=-110 
dBm/Hz and -40 dB hybrid suppression. 

3. Filtered multitone modulation 

This multicarrier modulation realization is sometimes called Filter bank Modulation 
Technique (Cherubini et al.2000). Figure 2 shows a FMT communication system realized by 
a critically sampled filter bank. The critically sampled filter bank, where the upsampling 
factor is equal to the size of filter bank 2N, can be realized efficiently with the help of the 
FFT algorithm, which will be described later. More concretely, the filter bank is non-critical, 
if the upsampling factor is higher than the size of filter bank (2N). 
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Fig. 2. Basic principle of Filtered MultiTone modulation. 
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The output signal x(n) of the FMT transmitter given in Fig. 2 can be described using relation: 

1 oo 2N-1 

*(») = -/=£ 2xfa(n-2kN) (2) 

V2A/ j := _ os ,-_o 

The polyphase FIR filters with the impulse response hi(n) are the frequency-shifted versions 
of low pass filter with impulse response h(n), called prototype filter: 

. 2nro 

h(n) = - l =h(n)e 2N (3) 



In equation (3) h(n) is the impulse response of the prototype FIR filter. The order of this 
filter is 2yN, where /is the overlapping factor in the time domain. 

For a perfect demodulation of received signal after transmission through the ideal channel 
the prototype filter must be designed such that for the polyphase filters the condition hold, 
which is expressed by the equation: 

Yh i {n%(n-2Nk) = S i _ i ,S k (4) 

n 

iorO<i,i'<2N-l and k = ..., -1, 0, 1, ... 

In equation (4) Si is the Kronecker delta function. The equation defining the orthogonality 
between the polyphase filters is a more general form of the Nyquist criterion (Cherubini et 
al.2000). For example, condition (4) of perfect reconstruction is satisfied in the case of DMT 
modulation, given in equation (1), because the sine spectrums of individual carriers have 
zero-values for the rest of corresponding carriers. The ideal frequency characteristic of the 
prototype filter to realize a non-overlapped FMT modulation system is given by the 
equation (5). 



JH(e^) 



1 1 

for </< — ,„ 

2T 2T ( 5 ) 

otherwise 



The prototype filter can be designed by the sampling the frequency characteristic (5) and 
applying the optimal window. Figure 3 shows the spectrum of an FMT modulation system 
for y = 10. The prototype filter was designed with the help of the Blackmen window. 
Suitable windows enabling the design of orthogonal filter bank are e.g. the Blackman 
window, Blackmanharris window, Hamming window, Hann window, flattop window and 
Nuttall window. Further examples of the prototype filter design can be found in (Cherubini 
et al., 2000) and (Berenguer & Wassel, 2002). 

The FMT realization of N - 1 carriers modulation system according to Fig. 4 needs 2N FIR 
filters with real coefficients of the order of 2yN. In (Berenguer & Wassel, 2002) a realization 
of FMT transmitter using the FFT algorithm is described. This realization is shown in Fig. 4. 
In comparison with DMT modulation, each output of IFFT is filtered additionally by an FIR 
filter hi(m) of the order of y. The coefficient of the Ji,(m) filter can be determined from the 
prototype filter h(n) of the order of 2yN: 

h i (m) = h(2mN + i) (6) 
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Fig. 3. FMT spectrum for y = 10 and Blackman window. 
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Fig. 4. The realization of FMT transmitter using FFT algorithm. 
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Fig. 5. The realization of FMT receiver using FFT algorithm. 
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The principle of FMT signal demodulation can be seen in fig 5. Since the individual carriers 
are completely separated, no ICI interference occurs. Equalization to minimize ISI 
interference can be performed in the frequency domain without the application of cyclic 
prefix (Benvenuto et al., 2002). Duplex transmission can be solved by both FDM and EC, 
without any further filtering, which is the case of DMT. If a part of the frequency band is 
shared (EC duplex method), the echo cancellation can be realized easily in the frequency 
domain. 

4. Overlapped FMT modulation 

The FMT modulation type mentioned in the previous section can be called non-overlapped 
FMT modulation. The individual carriers are completely separated and do not overlap each 
other. FMT realization of multicarrier modulation offers a lot of advantages, as mentioned in 
the preceding chapter. In particular, the frequency band provided is better utilized in the 
border parts of the spectrum designed for individual transmission directions, where in the 
case of DMT there are losses in the transmission rate. The out-of-transmission-band 
emission is eliminated almost completely. If we use the EC duplex method, the simpler 
suppression of echo signal enables sharing a higher frequency band. A disadvantage of FMT 
modulation is the increase in transmission delay, which increases with the filter order y. The 
FMT transmission delay is minimally y times higher than the transmission delay of a 
comparable DMT system and thus the filter order y must be chosen as a compromise. The 
suboptimal utilization of provided frequency band in the area between individual carriers 
belongs to other disadvantages of non-overlapped FMT modulation. Individual carriers are 
completely separated, but a part of the frequency band between them is therefore not 
utilized optimally, as shown in Fig. 6. The requirement of closely shaped filters by reason of 
this unused part minimization just leads to the necessity of the high order of polyphase 
filters. 



non-overlaped FMT with y= 14 
non-overlaped FMT with y=6 
overlaped FMT with y= 6 




Fig. 6. Comparison of PSD of overlapped and non-overlapped FMT. 
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The above advantages and disadvantages of the modulations presented in the previous 
section became the motivation for designing the half subchannel overlapped FMT 
modulation. An example of this overlapped FMT modulation is shown in Fig. 7. As the 
Figure shows, individual carriers overlap one half of each other. The side-lobe attenuation is 
smaller than 100 dB. For example, the necessary signal-to-noise ratio (SNR) for 15 bits per 
symbol QAM is approximately 55 dB. 




100 



120 



140 



Fig. 7. Overlapped FMT spectrum for y = 6 and Nuttall window. 

The ratio between transmitted total power in overlapped and non-overlapped FMT 
modulations of equivalent peak power are shown in Table 1. Suboptimal utilization of the 
frequency band occurs for smaller filter orders. As has been mentioned, a higher order of 
filters increases the system delay. The whole system delay depends on the polyphase filter 
order, number of carriers and delay, which originates in equalizers. 
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1 Polyphase filter order; 

2 Ratio between power of overlapped and non-overlapped FMT modulation; 

3 Used window; 

Table 1. Comparison of ratio between whole transmitted power of overlapped and non- 
overlapped FMT modulation for the same peak power. 
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The designed filter has to meet the orthogonal condition, introduced by equation (4). An 
efficient realization of overlapped FMT modulation is the same as that of non overlapped 
FMT modulation introduced in Fig. 4. The difference is in the design of the filter coefficients 
only. Polyphase filters can be of a considerably lower order than filters in non-overlapped 
FMT modulation, because they need not be so closely shaped in the transient part. The 
shape of individual filters must be designed so as to obtain a flat power spectral density 
(PSD) in the frequency band utilized, because it enables an optimal utilization of the 
frequency band provided. Figure 7 shows an example of such overlapped FMT modulation 
with y = 6. The ideal frequency characteristic of overlapped FMT prototype filter can be 
defined with the help of two conditions: 

|H(e^)|=0 for |/|<A 

( 7 ) 
|H(e^)| 2 + |H(eM/-V 2 ))| 2 =1 for lfl> l_ 

In the design of polyphase filters of very low order it is necessary to chose a compromise 
between both conditions, i.e. between the ripple in the band used and the stopband 
attenuation. Examples of some design results for polyphase filter orders of 2, 4 and 6 are 
shown in (Silhavy, 2008). The filter design method based on the prototype filter was 
described in the previous chapter. 

5. Equalization in overlapped FMT modulation 

In overlapped FMT modulation as well as in non-overlapped FMT modulation the inter- 
symbol interferences (ISI) occur even on an ideal channel, which is given by the FMT 
modulation system principle. Equalization for ISI interference elimination can be solved in 
the same manner as in non-overlapped FMT modulation with the help of DFE equalizers, in 
the frequency domain (see Fig. 5). 

If the prototype filter was designed to satisfy orthogonal condition (4), ICI interferences do 
not occur even in overlapped FMT modulation. More exactly, the ICI interference level is 
comparable with non-overlapped FMT modulation. This is demonstrated by the simulation 
results shown in Figures 8 and 9. 

In the Figures the 32-carrier system (Figure 8) and 256-carrier system (Figure 9) have been 
simulated, both with the order of polyphase filters y equal to 8. The systems were simulated 
on an CSA-mid loop in the 1MHz frequency band. From a comparison of Figures 8a and 8b 
it can be seen that ICI interferences in overlapped and non-overlapped FMT modulations 
are comparable. In the case of a smaller order of polyphase filters y the ICI interferences are 
lower even in overlapped FMT. Figure 9 shows the system with 256 carriers. It can be seen 
that the effect of channel and the ICI interferences are decreasing with growing number of 
subchannels. The level of ICI interferences is dependent on the order of polyphase filters y, 
the type of window used and the number of carriers. 

As has been mentioned, the channel equalization whose purpose is to minimize ISI 
interference can be performed in the frequency domain. Each of the EQn equalizers (see 
Figure 5.) can be realized as a Decision Feedback Equalizer (DFE). The Decision Feedback 
Equalizer is shown in Fig. 10. The equalizer works with complex values (Sayed, 2003). 
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Fig. 8. ICI suppression in overlapped FMT (a) and in non-overlapped FMT (b) modulation 
with N = 32, y= 8 and Nuttal window. 
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Fig. 9. ICI suppression in overlapped FMT modulation with N = 256, y = 8 and Nuttal 
window. 
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Fig. 10. Decision Feedback equalizer. 

The Decision Feedback Equalizer contains two digital FIR (Finite impulse response) filters 
and a decision circuit. The feedforward filter (FF) with the coefficients Wff and of the order 



Half-overlap Subchannel Filtered MultiTone Modulation and Its Implementation 



373 



of M is to shorten the channel impulse response to the feedback filter (FB) length R. The 
feedforward filter is designed to set the first coefficient of the shortened impulse response of 
channel to unity. With the help of the feedback filter (FB) with the coefficients 1- Wfb and 
order R we subtract the rest of the shortened channel impulse response. The whole DFE 
equalizer thus forms an infinite impulse response (IIR) filter. A linear equalizer, realized 
only by a feedforward filter, would not be sufficient to eliminate the ISI interference of the 
FMT system on the ideal channel. 



x()t) 



-4© — *(*) 



h k±>^ 

t 



w 



FF 



z(k)^ r(k) 



K±> 



x(k) 



n(k) 



t(k) 



1— W 



FB 



Fig. 11. Description of the sought MMSE minimization for the computation of FIR filters 
coefficients of equalizer 

The sought minimization of mean square error (MMSE) is described in Fig. 11. The 
transmission channel with the impulse response h includes the whole of a complex channel 
of the FMT modulation from Xk to Vk. The equalization result element r(k) is compared with 
the delayed transmitted element x(k). The delay A is also sought it optimizes the 
minimization and is equal to the delay inserted by the transmission channel and the 
feedforward filter. The minimization of the mean square error is described by equation (8). 



e(k) = x(k-A)- r(k) =x(k-A)- t(k) - z(k) = 

R-l 

= x(k-A)-x(k-A) + Y J x{k-n-A)-w VB (n)- £ y(k -n)-w w (n) 

n=0 

and iu FB (0) = l 



M-l 

I: 

ii -0 



(8) 



On the assumption of correct estimate and thus the validity of x(k-A) = x(k -A) we can 
simplify equation (8): 



M-l 

I 

ii -0 



e (k) = ^x{k -n - A) ■ w VB (n) - £ y(k-n)-w w (n) and w FB (0) = l 



The MMSE can be sought: 



m r 



EM 



: W FB R xx W FB - W FB R xyA W FF " W FF R yxA W FB + W FF R yy W FF 



(9) 



(10) 



The sought minimization mdfe-mmse under unity constraint on the first element of shortened 
response (Silhavy, 2007), introduced by equation (11), is shown by equation (12): 
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m DFE-MMSE - W FB^X | Y W FB + ^y- ~^0 W FB ) 



(11) 



'DFE-MMSE _ f J -l ( 

*0 K X|Y *0 



(12) 



where 



R X |y=Rx 



**-xyA**-yy "-yxA 



(13) 



and fo is the column zero vector of the same length as feedforward filter R with element 
one in the first position: f = [1 R _j] 

In this equation, R xx is the autocorrelation matrix of the input signal of M x M size, R xy A and 
RyxAare the correlation matrices between the input and output signals ofMxR and R x M 
sizes. These matrices are dependent on the parameter delay A, which defines the shift 
between the input and the output signals. For these matrices it holds that R 



xvA 



= R 



vxA 



During the computation it is necessary to find the optimal delay A op t, which can be 
determined based on computed minimization midfe-mmse- The optimal delay, however, must 
be set the same for the whole equalizer bank. During the heuristic optimal delay search for 
each equalizer the most frequent delay needs to be determined and this delay must be used 
for coefficient computation of each equalizer of demodulator. In opposite case, the 
individual symbols would be mutually shifted, which would cause demodulation 
difficulties. R yy is the autocorrelation matrix of the output signal of R x R size. This matrix 
includes the influence of the channel impulse response h and noise. 

The vectors of coefficients of equalizer for a given delay A can be computed as shown by the 
following equations: 



K X|Y r 

rf R "V" 

•0 1V X|Y r 



(14) 



= Ryy RyxA W FB 



(15) 



Alternatively, we can based the computation of the coefficients of individual DFE equalizer 
filters on channel impulse response h only, without including the input signal and noise n. 
This solution can be simply derived based on the previous solution after following 
derivation. The convolution matrix of a channel with impulse response h and length k is 
defined by the equation: 



H = 
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l Jc-l 
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(16) 
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From this matrix we chose the shortened part in the terms of delay A with the help of vector G: 

k +M-1 samples 

, A > 

G=diag(0, ... , 0, 1, ... , 1, 0, ..., 0) (17) 

a ~HT 

The sought matrix Rx|y , introduced by equation (13), of the Zero-forcing solution is shown 
by equation (18): 



R 



X I Y_ZeroForce 



= i gh(h h h) h h g h 



(18) 



The approach presented enables determining the individual equalizer filter coefficients from 
the channel impulse response h only. The matrix G determines the maximized area and thus 
the delay A between the input and the output signals. In some cassis the matrix Rx | Y_zeroForce 
is not invertible. This problem can be fixed by increasing the components in the main 
diagonal, as shows by the equation: 



A = A + k 



opt 



maxjdiag(A)} -I 



(19) 



where max{daig(A)} is the maximal element of the main diagonal of matrix A, and A: op t is the 
optimization factor, which can be increased in several steps (e.g. lO 10 , 10 -8 , lO 6 , ...). 
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Fig. 12. Achieved MSE in dependence on both filter order for overlapped FMT system with 
y =10, Nuttalwin window and ANSI 7 channel. 



376 



Discrete Time Systems 



For the purpose of analyses we choose an FMT system with N = 32 carriers, a sampling 
frequency of 2 MHz, the Nuttalwin window type and the polyphase filter order y =10, 12 
and 14. Selected tested channels were the ideal channel and ANSI 7. The channels were 
modeled by an infinite impulse response (IIR) filter of the order of 60. In Figures 12, 13 and 
14 for the purpose of comparing whole FMT systems the MSE is computed as summation of 
MSE of all N carriers. Figure 12 shows the dependence of achieved MSE on the order of both 
filters for overlapped FMT system with y =12, Nuttalwin window and ANSI 7 channel. The 
optimal combination of orders can be different for different FMT systems and channel types. 
From Figures 13 a) and 13 b) it can be seen that the transmit and receive filters are of 
primary influence on the appearance of ISI interference. The necessary equalizer filter order 
is a little higher than the polyphase filter order. 
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Fig. 13. Achieved MSE in dependence on filters order M=R for different FMT systems on 
ideal channel (a) and ANSI 7 channel (b). 
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6. Implementation of Filtered MultiTone in Matlab and on DSP 

This chapter is devoted to describtion of the implementation of FMT and DMT modulations. 
Theoretical properties of the FMT modulations with all advantages and disadvantages have 
been described in previous chapters. Implementation of FMT modulation in Matlab- 
Simulink and on DSP is necessary for comparison of characteristics of this modulation with 
other ones in various communication systems. 

Design of proper prototype filter is the first step in implementation of FMT. Prototype filter 
design methods are mentioned in chapter 3. Prototype filters for the implementation have 
been designed by the windowing method, using the Blackman, Modified Blackman and 
Nutall windows. The polyphase filters for non-overlap FMT modulation were of the order 
y =14 (Blackman window) and in half-overlap FMT modulation filters of the order y =6 
(Modified Blackman window, subchannel crossing at -3dB) and y =8 (Nuttall window, 
subchannel crossing at -6dB). 

As mentioned above, almost twice higher order of polyphase filters in the non-overlap FMT 
had to be used because of the need for a spectral separation of individual subchannels, i.e. 
the need for a sharp transition from pass-band to stop-band. Two variants of half- 
overlapped FMT were used for comparison of spectral characteristics and power spectral 
density. 



Blackman window, y=14 
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Fig. 14. Prototype filters used in implementation 

Furthermore, we proposed a suitable model of FMT Modulation in Matlab-Simulink. As a 
source of pseudorandom binary sequence the Bernoulli random binary generator was 
chosen. Binary data are transmitted to the bank of QAM modulators, number of bits per 
channel is determined by signal to noise ratio on carrier, for example by the help of a bit- 
loading algorithm (Akujobi & Shen, 2008). After it the IFFT modulation is performed. In the 
model, we consider transmission over metallic lines and also comparison of the FMT with 
DMT modulation, so it is necessary to complement the modulation symbols to the number 
2N according to (1) before IFFT. Filtering of IFFT output by the filter bank is the final step on 
the transmitter side. The filter bank was built-up from polyphase components of prototype 
filter mentioned above. 
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The procedure in the receiver is inverted. Firstly the received signal is filtered by receiver 
filter bank and after that, FFT is performed. In the next step the added symbols are removed 
i.e. the symbols and N...2N. Through the characteristics of FMT modulation mentioned in 
chapter 5 is necessary to use equalization. DFE equalization with RLS adaptive algorithm is 
used in our model. Equalized symbols are then demodulated by a bank of QAM 
demodulators. 
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Fig. 15. FMT system in Matlab-Simulink 

The resistance of narrowband noise on chosen carrier was tested on this model. This type of 
interference is very common in real conditions. Narrowband noise on 10 th carrier was 
applied in our case. 

The evaluation was done by measuring the signal to noise ratio, SNR. For the half-overlap 
FMT modulation the measurements were performed only for variants with subchannel 
crossing at level -3dB. The results of the measurements are presented in Fig. 17. It is obvious 
that the DMT modulation has the worst properties, where the narrowband interference on 
10 th carrier degrades SNR on a large number of surrounding carriers. The opposite case is 
FMT modulation, in both variants the SNR is degraded only on the carrier with narrowband 
interference. For the half-overlap FMT modulation degradation on two nearby carriers was 
expected, but the measurement shows degradation only on 10 th carrier. 

The model described above can be adjusted and implemented on DSP. The chosen 
development kit uses the TI C6713 floating-point digital signal processor. The model is 
divided into part of transmitter and part of receiver. The signal processing procedure is 
identical to the model. After generation of pseudorandom binary sequence, QAM 
modulation is performed. The number of bits transmitted on the sub-carrier is chosen before 
the actual implementation. After it the IFFT modulation is performed and each output is 
filtered by the transmitter filter bank. The last step is to adjust the amplitude of the 
transmitted signal to the range of DAC converter. In this way modified model was then 
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compiled and implemented on digital signal processor with the help of the Link for CCS 
toolbox. 

This way of generating code is fully functional and they allow measuring the proposed 
algorithm directly in the digital signal processor but they definitely cannot be considered 
optimized. It is convenient to use libraries that are optimized for a given processor and 
replace the standard Simulink blocks by optimized ones. It is also possible to replace the 
original number formats by formats corresponding to the processor. Also the filterbank can 
be designed in two ways. The first way is independent filtering in each branch of filterbank 
(Sysel, Krajsa 2010). 
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Fig. 16. Efficient filterbank implementation 

The second one is described on Fig. 16, where h™ is n-th coefficient of m-th filter, X' m is ;'-th 
IFFT symbol in m-th branch and o' m is ;'-th output sample in m-th branch. We have three 
buffers, one (a) for prototype coefficients, one (b) for input symbols from IFFT, and the last 
one (c) for output frame. Buffer b is FIFO buffer, samples are written in frames of 2N 
samples. This way of filtering is more effective, because we need only one for cycle for 
computing one output frame. 




Fig. 17. SNR for a) DMT, b) Non-overlapped FMT, c) Half-overlapped FMT with 
narrowband noise 
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In the term of testing and comparing the implementation on DSP is interesting for the 
possibility of power spectral density measurement and for its characteristics inside and 
outside of the transmission band of partial subchannels on real line. In Fig. 19 is measured 
PSD for the considered modulations. 
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Fig. 18. FMT transmitter adjusted for implementation 
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Fig. 19. Measured power spectral densit 

It is clear that the implementation results confirm the theoretical assumptions about the 
properties of implemented modulations, mainly about their spectral properties. For the half- 
overlap FMT modulation the PSD measured was flat, as well as with DMT modulation, but 
the side lobes are suppressed by up to 50 dB. For the non-overlap FMT modulation perfectly 
separated subchannels and strongly repressed side lobes are again evident. 
In the implementation the computational complexity of individual modulation was also 
compared. The most common form of DMT modulation needs to implement only the 2N- 
point FFT, while with FMT each FFT output must be filtered. This represents an increase in 
the required computational power and in the memory used. A comparison of DMT and 
FMT for different systems is shown in the table. It compares the number of MAC 
instructions needed for processing one frame of length 2N. 
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7. Conclusion 

Based on a comparison of DMT and non-overlapped FMT multicarrier modulations we 
introduced in this contribution the half-overlap subchannel FMT modulation. This 
modulation scheme enables using optimally the available frequency band, such as DMT 
modulation, because the resultant power spectral density of the signal is flat. Also, the 
border frequency band is used optimally, the same as in non-overlapped FMT modulation. 
Compared to non-overlapped FMT modulation the subchannel width is double and the 
carriers cannot be too closely shaped. That enables using a smaller polyphase filter order 
and thus obtaining a smaller delay. In section 5 we demonstrated that if the prototype filter 
was designed to satisfy the orthogonal condition, even in overlapped FMT modulation the 
ICI interferences do not occur. Furthermore, a method for channel equalization with the 
help of DFE equalizer has been presented and the computation of individual filter 
coefficients has been derived. 
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1. Introduction 

Discrete multitone (DMT) is a digital implementation of the multicarrier transmission 
technique for digital subscriber line (DSL) standard (Golden et al., 2006; Starr et al., 1999). 
An all-digital implementation of multicarrier modulation called DMT modulation has been 
standardised for asymmetric digital subscriber line (ADSL), ADSL2, ADSL2+ and very high 
bit rate DSL (VDSL) (ITU, 2001; 2002; 2003). ADSL modems rely on DMT modulation, 
which divides a broadband channel into many narrowband subchannels and modulated 
encoded signals onto the narrowband subchannels. The major impairments such as the 
intersymbol interference (ISI), the intercarrier interference (ICI), the channel distortion, echo, 
radio-frequency interference (RFI) and crosstalk from DSL systems are induced as a result 
of large bandwidth utilisation over the telephone line. However, the improvement can be 
achieved by the equalisation concepts. A time-domain equaliser (TEQ) has been suggested 
for equalisation in DMT-based systems (Bladel & Moenclaey, 1995; Baldemair & Frenger, 2001; 
Wang & Adali, 2000) and multicarrier systems (Lopez- Valcarce, 2004). 

The so-called shortened impulse response (SIR) which is basically the convolutional result 
of TEQ and channel impulse response (CIR) is preferably shortened as most as possible. By 
employing a TEQ, the performance of a DMT system is less sensitive to the choice of length 
of cyclic prefix. It is inserted between DMT symbols to provide subchannel independency 
to eliminate intersymbol interference (ISI) and intercarrier interference (ICI). TEQs have been 
introduced in DMT systems to alleviate the effect of ISI and ICI in case that the length of SIR 
or shorter than the length of cyclic prefix (F-Boroujeny & Ding, 2001). The target impulse 
response (TIR) is a design parameter characterising the derivation of the TEQ. By employing 
a TEQ, the performance of a DMT system is less sensitive to the choice of length of the cyclic 
prefix. In addition to TEQ, a frequency-domain equaliser (FEQ) is provided for each tone 
separately to compensate for the amplitude and phase of distortion. An ultimate objective of 
most TEQ designs is to minimise the mean square error (MSE) between output of TEQ and 
TIR which implies that TEQ and TIR are optimised in the MSE sense (F-Boroujeny & Ding, 
2001). 

Existing TEQ algorithms are based upon mainly in the MMSE-based approach (Al-Dhahir 
& Cioffi, 1996; Lee et al., 1995; Yap & McCanny, 2002; Ysebaert et al., 2003). These include 
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the MMSE-TEQ design algorithm with the unit tap constraint (UTC) in (Lee et al., 1995) and 
the unit energy constraint (UEC) in (Ysebaert et al., 2003). Only a few adaptive algorithms 
for TEQ are proposed in the literature. In (Yap & McCanny, 2002), a combined structure 
using the order statistic normalised averaged least mean fourth (OS-NALMF) algorithm for 
TEQ and order statistic normalised averaged least mean square (OS-NALMS) for TIR is 
presented. The advantage of a class of order statistic least mean square algorithms has been 
presented in (Haweel & Clarkson, 1992) which are similar to the usual gradient-based least 
mean square (LMS) algorithm with robust order statistic filtering operations applied to the 
gradient estimate sequence. 

The purpose of this chapter is therefore finding the adaptive low-complexity time-domain 
equalisation algorithm for DMT-based systems which more robust as compared to existing 
algorithms. The chapter is organised as follows. In Section 2 , we describe the overview of 
system and data model. In Section 3 , the MMSE-based time-domain equalisation is reviewed. 
In Section 4 , the derivation of normalised least mean square (NLMS) algorithm with the 
constrained optimisation for TEQ and TIR are introduced. We derive firstly the stochastic 
gradient-based TEQ and TIR design criteria based upon the well known low-complexity 
NLMS algorithm with the method of Lagrange multiplier. It is simple and robust for ISI and 
ICI. This leads into Section 5 , where the order statistic normalised averaged least mean square 
(OS-NALMS) TEQ and TIR are presented. Consequently, the adaptive step-size order statistic 
normalised averaged least mean square (AS-OSNALMS) algorithms for TEQ and TIR can be 
introduced as the solution of MSE sense. This allows to track changing channel conditions and 
be quite suitable and flexible for DMT-based systems. In Section 6 , the analysis of stability 
of proposed algorithm for TEQ and TIR is shown. In Section 7 and Section 8 , the simulation 
results and conclusion are presented. 

2. System and data model 

The basic structure of the DMT transceiver is illustrated in Fig. 1. The incoming bit stream 
is likewise reshaped to a complex-valued transmitted symbol for mapping in quadrature 
amplitude modulation (QAM). Then, the output of QAM bit stream is split into N parallel bit 
streams that are instantaneously fed to the modulating inverse fast Fourier transform (IFFT). 
After that, IFFT outputs are transformed into the serial symbols including the cyclic prefix 
(CP) between symbols in order to prevent intersymbol interference (ISI) (Henkel et al., 2002) 
and then fed to the channel. The transmission channel will be used throughout the chapter is 
based on parameters in (ITU, 2001). The transmitted signal sent over the channel with impulse 
response is generally corrupted by the additive white Gaussian noise (AWGN). 
The received signal is also equalised by TEQ. The number of coefficients of TEQ is particularly 
used to make the shortened-channel impulse response (SIR) length, which is the desired 
length of the channel after equalisation. The frequency-domain equaliser (FEQ) is essentially 
a one-tap equaliser that is the fast Fourier transform (FFT) of the composite channel of 
the convolution between the coefficients of the channel (h) and the tap-weight vector (w) 
of TEQ. The parallel of received symbols are eventually converted into serial bits in the 
frequency-domain. 

The data model is based on a finite impulse response (FIR) model of transmission channel 
and will be used for equaliser in DMT-based systems. The basic data model is assumed that 
the transmission channel, including the transmitter and receiver filter front end. This can 
be represented with an FIR model h. The /c-th received sample vector which is used for the 
detection of the A:-th transmitted symbol vector xj. N , is given by 



Adaptive Step-size Order Statistic LMS-based 
Time-domain Equalisation in Discrete Multitone Systems 



385 





























P/S 


QAM 




S/P 




IFFT 




CP 








CP 



QAM 
output 



Fig. 1. Block diagram for time-domain equalisation. 



Vk,l+A 
.Vk.N-l+A- 

yt,/+A:N-l + A 




(1) 



where 



The notation for the received sample vectors y k i+a-n-I+a anc ^ me received samples ffk,i+A 
are introduced by 



Yk,l+A:N-l+A - I Vk,l+A ' ' ' Dk,N-l+A ] , 



(2) 



where / determines the first considered sample of the A:-th received DMT-symbol and 
depends on the number of equaliser taps L. The parameter A is a synchronisation delay. 

h is the CIR vector h with coefficients in reverse order. 

I is an n x n identity matrix and ® denotes the Kronecker product. The (N + v) x N matrix 
V v , which adds the cyclic prefix of length v , is introduced by 



x *r,-i/:N-l 



°vx(N-v) |If 



x ;c,0:N-l/ 



(3) 



p» 



where the sample vector x^. _ v: _i is called a cyclic prefix (CP). 

fH = jr*j is the N x N IDFT matrix. 

The Nxl transmitted symbol vector xj. ;v is introduced by 

iT 



x k,N 



x k,0 ■ ' ' x k,N-\ 



x k,N-l 



k,^+l 



(4) 



The vector »7jt/+A:N-l+A is a sample vector with additive channel noise, and its 
autocorrelation matrix is denoted as Y2l = E^^r^}. 

The matrices 0m\ and 0( 2 ) m Eq.(l) are the zero matrices of size (N — I) X (N — L + 2v + 
A + /) and (N - /) x (N + v - A), respectively. 

The transmitted symbol vector is denoted as Xk-l-.k+l,N / where Xk-1,N and x/ c+ j jy introduce 
ISI. The Xk,N is the symbol vector of interest. 
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Fig. 2. Block diagram of MMSE-TEQ. 

Some notation will be used throughout this chapter as follows: £{■}, (•) / (') H denote as 
the expectation, transpose and Hermitian operators, respectively. The vectors are in bold 
lowercase and matrices are in bold uppercase. 

3. Minimum mean square error-based time-domain equalisation 

The design of minimum mean square error time-domain equalisation (MMSE-TEQ) is based 
on the block diagram in Figure 2. The transmitted symbol x is sent over the channel with the 
impulse response h and corrupted by AWGN r\. The convolution of the L-tap TEQ filter 
w and the CIR h of N/, + 1 samples are sufficiently shortened so that overall of impulse 
response has length v + 1 that should make TEQ as a channel shortener c = h * w, called 
the shorten impulse response (SIR). Then the orthogonality between the tones are restored 
and ISI vanishes (Melsa et al., 1996). 

The result of time-domain error e between the TEQ output and the TIR output is then 
minimised in the mean-square sense as 



min E{|e| 2 } = min E{|y T w - xjb| 2 } 



w,b 



min w J E 2 w + b J E 2 b - 2b J Exy(A)w, 

w,b ' 



(5) 
(6) 



where E 2 = E{yy T } and E 2 = E{xx T } are autocorrelation matrices, and where Exy(A) = 
£{x^y } is a cross-correlation matrix. 

To avoid the trivial all-zero solution w = 0, b = 0, a constraint on the TEQ or TIR is therefore 
imposed. 
Some constraints that are added on the TEQ and TIR (Ysebaert et al., 2003) as follows. 

1. The unit-norm constraint (UNC) on the TIR 
By solving Eq.(6) subject to 

b T b = l. (7) 

The solution of b is the eigen-vector and w can be given as 



E^b. 



(8) 



2. The unit-tap constraint (UTC) on the TEQ 

A UTC on w can be calculated with the method of the linear equation 



T 

e.- w 



1 or e, w 



1 



(9) 
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where e, is the canonical vector with element one in the j'-th position. By determining 
the dominant generalised eigen-vector, the vector w can be obtained as the closed-form 
solution 

A" 1 e, 



■s 



efA-V 



(10) 



whereA = E2-Ej y (E2)-iE xy . 

The unit-tap constraint (UTC) on the TIR 
Similarly, a UTC on b can be described as 

ejb= 1 or e ; T b= -1, (11) 



After computing the solution for b as 



-s 



eJA^ej 



(12) 



The coefficients of TEQ w can be computed by Eq.(8). 

4. The unit-energy constraint (UEC) on TEQ and TIR 
Three UECs can be considered as 

w T E^w = 1 or b T E^b = 1 or w T E^w = 1 & b T E^b = 1. (13) 

It has been shown that each of all constraints results in Eq.(13), which can be incorporated 
into the one-tap FEQs in frequency domain (Ysebaert et al., 2003). 

Most TEQ designs are based on the block-based computation to find TIR (Al-Dhahir & Cioffi, 
1996; F-Boroujeny & Ding, 2001; Lee et al., 1995), it will make high computational complexity 
for implementation. However, this algorithm has much better performance and is used for 
the reference for on-line technique. 

4. The proposed normalised least mean square algorithm for TEQ and TIR 

We study the use of the LMS algorithm by means of the simplicity of implementation 
and robust performance. But the main limitation of the LMS algorithm is slow rate of 
convergence (Diniz, 2008; Haykin, 2002). Most importantly, the normalised least mean square 
(NLMS) algorithm exhibits a rate of convergence that is potentially faster than that of the 
standard LMS algorithm. Following (Haykin, 2002), we derive the normalised LMS algorithm 
for TEQ and TIR as follows. 

Given the channel-filtered input vector y(n) and the delay input vector d(n), to determine the 
tap-weight vector of TEQ w(n + 1) and the tap-weight vector of TIR b(n + 1). So, the change 
Sw(n + 1) and Sb(n + 1) are defined as 

Sw(n + 1) = w(n + l) — w(n), (14) 

<5b(n + l)=b(n + l)-b(n) / (15) 



and subject to the constraints 



w H (n + l)y(n) = ? 1 (n), (16) 

b H (n + l)d(n) = ? 2 (n), (17) 
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where e(n) is the estimation error 

e(n) = w H (w + 1) y(n) - b H (n + 1) d(n) . (18) 

The squared Euclidean norm of the change 6vr(n + 1) and Sb(n + 1) may be expressed as 

M-l 
||^w( M + l)|| 2 = £ IwtCn + lJ-wtCn)! 2 , (19) 

fc=0 
M-l 

||Jb(« + l)|| 2 = X] |b t (n + l) -b fc (n)| 2 . (20) 

i-=o 

Given the tap-weight of TEQ Wj-(n) and TIR bj.(n) for fc = 0, 1, . . . ,M — 1 in terms of their real 
and imaginary parts by 

w k (n) = a k (n)+jb k (n), (21) 

b k {n) = u k (n) + j v k {n) . (22) 

The tap-input vectors y(«) and d(n) are defined in term of real and imaginary parts as 

y(n)=yi(n)+;y 2 (n), (23) 

d(n) = di(n)+;d 2 (n). (24) 

Let the constraints gi(n) and ,g"2( M ) be expressed in terms of their real and imaginary parts as 

§i(n) = gia{n) + j gw(n) , (25) 

gl{n) = g2a{n) + ] g 2b {n) . (26) 

To rewrite the complex constraint of Eq.(16) as the pair of real constraints 

M-l 

gi(«) = E [w t (n + i)] H y(«) 

/c=0 
M-l 

= E {K(n + i)+;^(« + i)]*[yi(n-fc)+;y 2 («-fc)]} 

Jfc=0 

M-l (27) 

= E iM" + i)yi(« - fc) + bt(n + i)y 2 (n - fc)] 

/c=0 

+ j [a t (» + l)y 2 (« - fc) - &*(« + l)yi(" - fc)]} 
= gu(n) + j gib(n) ■ 



Therefore, 



M-l 

g\a{n)= £ [a k (n + l) yi (n-k) + b k (n + l)y 2 (n-k)} , (28) 

*:=0 

M-l 

gib(n)= E [a k (n + l)y 2 (n - k) - b k (n + l) yi (n - k)} . (29) 

fc=o 



Adaptive Step-size Order Statistic LMS-based 

Time-domain Equalisation in Discrete Multitone Systems 389 

To formulate the complex constraint of Eq.(17) as the pair of real constraints. 
M-l 

a(n)= E [b,(n + i)] H d(«) 

;c=o 

M-l 

= £ {[«jfc(n + l)+;ok(n + l)]*[di(n-fc)+yi 2 (n-fc)]} 

J:=0 

M^i (30) 

= E {K(« + l)di(M-fc)+^(M + l)d 2 ("-fc)] 

J:=0 

+ ;' [«*(« + l)d 2 (n - k) - v k (n + V)d x (n - k)]} 

= g2a(n) +jg2b(n) ■ 
Therefore, 

M-l 

g2a(n)= E [Uk(n + l)di(n-k)+v k (n + l)d 2 (n-k)] , (31) 

fc=0 

M-l 

S2*(n)= E [«t(n + l)i 2 («-*)-fjfc(« + 1)^1 (»-*)] • (32) 

k=0 

4.1 The proposed normalised least mean square time-domain equalisation (NLMS-TEQ) 

We define the real-valued cost function ]\(n) for the constrained optimisation using Lagrange 
multiplier. 1 (Haykin, 2002) 

h(n) = |j Sv/(n + 1)|| 2 + Aj { [a k (n + l) Vl (n -k) + b k {n + l)y 2 {n - k)] - g la {n)} 
+ A 2 { [a k (n + l)y 2 (n-k)-b k (n + l)yi(n-k)] - gw{n)} 

M-l 

= E { [«*(» + 1) - «*(»)] 2 + [**(« + 1) - &*(»)] 2 } 

,c=0 (33) 

M-l 

+ Ai { E fot(« + l)yi(« - fc ) + M» + l)y2(« - *)] - Sm(n)} 

M-l 

+ A 2 { E l a k(n + l)y 2 (n-k)-b k (n + l)y 1 (n-k)] -gib(n)}/ 

*:=0 

where Aj and A 2 are Lagrange multipliers. We find the optimum values of u k (n + 1) and 
b k (n + 1) by differentiating the cost function ]\(n) with respect to these parameters and set 
the both results equal to zero. Hence, 

a /i(") _ 



da k (n + 1) 



1 The method of Lagrange multiplier is defined as a new real-valued Lagrange function h{w) 

h(w) = f{w) + AjRe [C(w)] + A 2 Im [C(w)\ 

where f[to) is the real function and C(w) is the complex constraint function. The parameters Aj and A 2 
are the Lagrange multipliers, where A = Aj + j A 2 
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and 

3/l(») 



0. 



db k (n + l) 

The results are given by 

2 [a k (n + 1) - a k (n)] + A 1 y 1 (n - fc) + A 2 y 2 (n - fc) = , (34) 

2 [^(« + l)-^(M)]+A 1 y 2 (M-fc)-A 2 yi(n-fc) =0. (35) 

From Eq.(21) and Eq.(23), we combine these two real results into a single complex one as 

a/i(«) a/i(n) , ,. aji(«) . ,„,, 



(37) 



3w t ,(;i + 1) da k (n + 1) ; db k (n + 1) 
Therefore, 

3w/(n + 1) = {2 [fl,c(n + x) " fl,c(n)] + Al3/l(M ~ k) + Xm{n ~ k)}+ 

j {2 [b k {n + 1) - b k (n)\ + \ lV2 {n - k) - \ 2Vl (« - *)} 
= 2 [« t (n + l)+;l» fc (H + l)]-2 [«jt(n)+;6i(n)] + 

Aj [yi(n-k)+jy 2 (,n-k)] - ; A 2 [y a (n - fc) +jy 2 (n -k)] 

= 2 [a t (n + l)+;t fc (n + l)]-2 [a k (n) +jb k (n)\ + 

(A a -;' A 2 ) [yj(« - k)+jy 2 {n - k)\ 
= 0. 

Thus, we get 

2 [w fc (n + l)-w t (n)]+A^y(n-fc) = 0, for k = 0,1,. . .,M- 1 (38) 

where A l( , is a complex Lagrange multiplier for TEQ as 

A w = X 1 +j\ 2 . (39) 

In order to find the unknown A*„, we multiply both sides of Eq.(38) by y* (n — k) and then sum 
over all integer values of k for to M — 1. Thus, we have 

2 [w t (n + l)-w t (n)] y*(n-fc) = -A* y(n - fc)y*(n - fc) 

M-l M-l 

2 £ [w,(n + l) y *(n-fc)-w fc (n)y*(n-/c)] = -A*, E ly("-fc)| 2 
k=0 k=0 



[w r (« + l)y»-w T (n)y»] = -A*Jy(«)| 



2 



Therefore, the complex conjugate Lagrange multiplier A*„ can be formulated as 

X ™=Tt\u2 [w T (n + l)y*(n)-w T (n)y*(n)l , (40) 

lly(«)lr L J 
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where ||y(«) || is the Euclidean norm of the tap-input vector y(n). 

From the definition of the estimation error e(n) in Eq.(18), the conjugate of e(n) is written as 

e*(n) =w T (M + l)y*(M)-b r (M + l)d*(n). (41) 

The mean-square error | e(n) \ 2 is minimised by the derivative of | e(n) \ 2 with respect to w(w + 
1) be equal to zero. 



d\e(n)\ 2 
3w(n + l) 

Hence, we have 



[w H (n + l)y(«)-b H (n + l)d(n)] y*(n)=0. (42) 



w H (n + l)y(n) = b H (n + 1) d(n) , (43) 

and the conjugate of Eq.(43) may expressed as 

w T (n + l)y*(n) =b T (n + l)d*(n). (44) 

To substitute Eq.(44) and Eq.(41) into Eq.(40) and then formulate A^ as 

A;=^ w e*(«). (45) 

l|y(«)lr 

We rewrite Eq.(38) using Eq.(14) by writing, 

2^w(« + l) = -A*„y(H) (46) 

The change Sw(n + 1) is redefined by substituting Eq.(45) in Eq.(46). We thus have 

M»+i)=|j^py(») '*(»)• (47) 

To introduce a step-size for TEQ denoted by \i w and then we may express the change 5vr(n + 
l)as 

S w (n + l) = ^-^y(n)e*(n). (48) 

We rewrite the tap-weight vector of TEQ vr(n + 1) as 

w(n + 1) = w(n) + 5vr(n + 1) . (49) 

Finally, we may obtain the tap-weight vector of TEQ w(w + 1) in the well-known NLMS 
algorithm. 

w(n + l) =w(m)- „ p°—; y(n)e*(n). (50) 

llywlr 

where e*(n) is described in Eq.(41). 
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4.2 The proposed normalised least mean square-target Impulse response (NLMS-TIR) 

We formulate the real-valued cost function ] 2 (n) for the constrained optimisation problem 
using Lagrange multiplier. 

} 2 {n) = || Sb{n + 1)|| 2 + A 3 { [u k (n + \)& x {n - k) + v k (n + l)d 2 (n - k)] - g 2a (n)} 
+ A 4 { [u k (n + l)d 2 (n - k) - v k (n + \)d x {n - k)\ - g 2b {n)} 

M-l 

= £ { [u k (n + 1) - u k (n)] 2 + [v k (n + 1) - v k (n)] 2 } 

,c=0 (51) 

M-l 

+ A 3 { £ [u k (n + l)d 1 {n-k) + v k (n + l)d 2 (n-k)}-g2 a (n)} 
k=0 
M-l 

+ A 4{ I] [u k (n + l)d 2 {n - k) - v k (n + l)di(n - k)] - g 2b (n)} , 

k=0 

where A3 and A4 are Lagrange multipliers. We find the optimum values of u k (n + 1) and 
v k (n + 1) by differentiating the cost function J 2 (n) with respect to these parameters and then 
set the results equal to zero. Hence, 

3/2(") _ Q/ 



du k (n + l) 
and 



'dh(n) _ 



dv k (n + 1 
The results are 

2 [u k (n + 1) - M)fc(n)] + A 3 di(n - fc) + A 4 d 2 (« - Jfc) = , (52) 

2 [^(n + 1) - v k (n)} + \ 3 d 2 (n - k) - A 4 d 1 (« - fc) = . (53) 

From Eq.(22) and Eq.(24), we combine these two real results into a single complex one as 

3/2(n) 9/2(n) |; . 3/2(n) _ Q (M) 



Therefore, 

3/ 2 (n) 



3b t (H + l) 



9b, c (n + l) du k (n + l) ' dv k (n + l) 



{2 [m 4 (m + 1) - u k (n)\ + A 3 d x (n - fc) + A 4 d 2 (w - ^)}+ 

7 {2 [^(h + 1) - ^(n)] + \ 3 d 2 {n - k) - X A d x {n - k)} 
2 [ Uk {n + l)+jv k (n + l)]-2 [u k (n) + 7 v k (n)] + 

A 3 [d t (n - k)+jd 2 {n - k)} -7A4 [rfj(n - k) + j d 2 {n - k)} 
2 [u t (n + 1) + 7 » fc (n + 1)] - 2 [ U]t (n) + 7 Cjt(n)] + 

(A 3 -7A4) [dj(n - k)+jd 2 {n - k)\ 
0. 



(55) 
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Thus, we have 

2 [b k {n + l)-b k (n)]+\* b d(n-k) = 0, for k = 0,1,. . .,M - 1 (56) 

where Aj, is a complex Lagrange multiplier for TIR 

k h = A 3 + j A 4 (57) 

To multiply both side of Eq.(56) by d* (ft — fc) to find the unknown A? and then sum over all 
possible integer values of k for to M — 1. Thus, we get 

2 [bfc(n + l)-bjt(n)] d*(n-fc) = -A£ d(« - fc) d* (n - k) 

M-l M-l 

2 £ [bi(n + l)d*(n-fc)-b k (n)d*(n-fc)] = -^ £ |d(n-fc)| 2 

fc=0 k=0 

2 [b T (n + l)d*(n)-b r (n)d*(n)] = -A£||d(«)|| 2 

Therefore, 



— ^ [b T (ft + l)d*(«)-b T (ft)d*(ft)] . (58) 



-2 

where 1 1 d ( n ) 1 1 is the Euclidean norm of the tap-input vector d ( n ) . 
To substitute Eq.(41) and Eq.(44) into Eq.(58) and then formulate Af as 

K = wuw e * {n) - (59) 

We rewrite Eq.(56) using Eq.(15) by 

2<5b(« + l) = A£d(n) (60) 

To redefine the change 8b (n + 1) by substituting Eq.(59) in Eq.(60). We thus get, 

1 

To introduce a step-size for TIR \i^ and then we redefine the change 5b(n + 1) simply as 

n 

l|d(«)|| 2 

where pii, is the step-size for the NLMS-TIR. 

We rewrite the tap-weight vector of TIR b(« + 1) as 

b(n + l)=b(n) + 5b(n + i). (63) 

Finally, we may formulate the tap-weight vector of TIR b(w + 1) in the normalised LMS 
algorithm. 

b(n + l)=b(n)+ J' b d(n)e*(n), (64) 

where e* (n) is given in Eq.(41). 

To comply with the Euclidean norm constraint, the tap-weight vector of TIR b(n + 1) is 

normalised as 



5b(n + l) = MJ , 2 d(n)e*(n). (61) 



5b(n + l)= „" 2 d(»)e*(n), (62) 
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5. Adaptive step-size order statistic-normalised averaged least mean 
square-based time-domain equalisation 

Based on least mean square (LMS) algorithm, a class of adaptive algorihtms employing order 
statistic filtering of the sampled gradient estimates has been presented in (Haweel & Clarkson, 
1992), which can provide with the development of simple and robust adaptive filter across a 
wide range of input environments. This section is therefore concerned with the development 
of simple and robust adaptive time-domain equalisation by defining normalised least mean 
square (NLMS) algorithm. 

Following (Haweel & Clarkson, 1992), we present the NLMS algorithm which replaces linear 
smoothing of gradient estimates by order statistic averaged LMS filter. A class of order statistic 
normalised averaged LMS algorithm with the adaptive step-size scheme for the proposed 
NLMS algorithm in Eq.(50) and Eq.(64) that are shown as (Sitjongsataporn & Yuvapoositanon, 
2007). 

w(n + l)=w(n)- ! ll " ( \ ) n? M w a w , (66) 

llywlr 

b(n + l)=b(n) + -^LM b a b , (67) 

l|d(n)|| z 

with 

Mm = T{ e*(n)y(n), t{n - l)y(n - 1),. . . , t(n - N w + l)y(n - N w + 1)} , (68) 

M b = T{e*(n)d(n), e*{n - l)d(n - 1),... , e*{n -N b + l)d(n- N b + 1)} , (69) 



■Ji 



-H 



and 



e(n) = w H (n)y(n)-b (n)d(n) , (70) 

a l „=[a w (l),a w (2),...,a l „(N l „)\, a w {i) = 1/N W ; i = 1,2, . . . ,N W . (71) 

a b =[a b (l),a b (2),...,a b (N b )}, a b (j) = l/N b ; j = 1,2, . . .,N b . (72) 

where e(ri) is a priori estimation error and T{-} operation denotes as the algebraic ordering 
transformation. The parameters a w and a b are the average of the gradient estimates of 
weighting coefficients as described in (Chambers, 1993). The parameters }iw(n) and }i b (n) 
are the step-size of w(n) and b(n). The parameters N w and N b are the number of tap-weight 
vectors for TEQ and TIR, respectively. 

Following (Benveniste et al., 1990), we demonstate the derivation of adaptive step-size 
algorithms of fi w (n) and Ji b (n) based on the proposed NLMS algorithm in Eq.(50) and Eq.(64). 
The cost function / m ;„(«) may be expressed as 

J mln (n) = minE{\e(n)\ 2 }, (73) 

w,b 

e (n) = w h (m + 1) y(n) - b H (n + 1) d(n) . (74) 

We then form the stochastic approximation equations for ii w (n + l) and ji b (n + 1) as (Kushner 
& Yang, 1995) 

]i w {n + 1) =}i w (n) + ix w {-\7J min (ii w )} , (75) 

m,(n + l) =M n ) + a b{-^ Jmin(Ub)} , (76) 



Adaptive Step-size Order Statistic LMS-based 
Time-domain Equalisation in Discrete Multitone Systems 



395 



x(n) 



AWGN + NEXT 



CIR 
h 



delay 
* A 



-^f^fi 



d(n) 



T 
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b 
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Fig. 3. Block diagram of adaptive step-size order statistic normalised averaged least mean 
square (AS-OSNALMS) TEQ and TIR. 



where V/ m , n (;( m ) and V /„„•„(;(;,) denote as the value of the gradient vectors. The parameters 

0i w and a b are the adaptation constant of pt w and ji^, respectively. 

By differentiating the cost function in Eq.(73) with respect to \i w and ]i b , we get 



djiw 

o>min 
d}l b 



VJminiHw) =e{n)y T (n)Y w , 



VJ mm (pi b ) = -e(n)d 1 (n)V b , 



(77) 
(78) 



where Y w = ^ and Y;, = -}"' are the derivative of w(n + 1) in Eq.(50) with respect to 
}i w {n) and of b(n + 1) in Eq.(64) with respect to Ji b (n) (Moon & Stirling, 2000). 
By substituting Eq.(77) and Eq.(78) in Eq.(75) and Eq.(76), we get the adaptive step-size Ji w (n) 
and }i b (n) as 



fi w (n + 1) =\i w (n) - a. l0 {e{n) y T (n)W w } , 
H b (n + 1) =}i b {n) + ct b {e(n) d T (n)Y b } , 



where 



T w (n + 1) 
T 6 (n + 1) 



y(») 



llyWII 2 

d(n) 

||d(«)||2 



}i w (n)y r (n) 

Hb(n)d T (n) 



lly(«)lr 



Then, we apply the order statistic scheme in Eq.(81) and Eq.(82) as 



Y«p(n + 1) 
Y 6 (n+1) 



y(n) 
lly(«)ll 2 

d(«) 

||d(n)||2 



fi a (n)y T (n) 

Hb(n)d T (n) 



Y w (n) - 



M w a w 

lly(«)ll 2 

M b a b 
||d(n)||2' 



(79) 
(80) 

(81) 
(82) 

(83) 
(84) 



396 Discrete Time Systems 

where M w , M b , a w , a b and e(n) are given in Eq.(68)-Eq.(72). 

6. Stability analysis of the proposed AS-OSNALMS TEQ and TIR 

In this section, the stability of the proposed AS-OSNALMS algorithm for TEQ and TIR are 
based upon the NLMS algorithm as given in (Hay kin, 2002). This also provides for the optimal 
step-size parameters for TEQ and TIR. 

According to the tap-weight estimate vector w(n) and b(w) computed in Eq.(66) and Eq.(67), 
the difference between the optimum tap-weight vector w '' f and w(n) is calculated by the 
weight-error vector of TEQ as 

Aw(n)=w°l*-w(B), (85) 

and, in the similar fashion, the weight-error vector of TIR is given by 

Ab(n)=b°P t -b(n), (86) 

By substituting Eq.(66) and Eq.(67) from w°'' f and b° pt , we have 

Aw(» + l)=Aw(»i) + -^| M w a w , (87) 

lly(«)lr 

where M w and a w are defined in Eq.(68) and Eq.(71). 

|d(n)|| 

where M}, and a b are given in Eq.(69) and Eq.(72). 

The stability analysis of the proposed AS-OSNALMS TEQ and TIR are based on the mean 

square deviation (MSD) as 

V w (n) = £{||Aw(«) || 2 }, (89) 

2? b (n) = E{||Ab(n)|| 2 }, (90) 

where V w (n) and T> b (n) denote as the MSD on TEQ and TIR. 

By taking the squared Euclidean norms of both sides of Eq.(87) and Eq.(88), we get 

||Aw(n + l)|| 2 =||Aw(n)|| 2 + 2/^ I Aw H ( M )-(A4 Iu a IU ) 

lly(«)lr 



Ab(n + 1) = Ab(n)- ^ AI J n2 M b a b , (88) 



(91) 



?4(n) {M l0 a l0 ) H {M w a w ) 
||y(n)||2 ||y(n)p 

||Ab(/7 + l)|| 2 =||Ab(n)|| 2 -2^M I Ab H (n)-(^ fe fl 6 ) 

}i 2 b {n) {M b a b ) H {M b a b ) 
\\d(n)\\ 2 \\d{n)\\ 2 ' l ; 

Then taking expectations and rearranging terms with Eq.(89) and Eq.(90), the MSD of w(n) is 
defined by 

V w (n + l) = V w (n)+2ii w (n)E{K(A W H (n)Z w (n))} 

+ ]ii(n)E{Vt(Z%(n)Z w (n))}, (93) 
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where £ w { n ) is given by 



£n,(n) = E 



M w a-g 

lly(«)ll : 



(94) 



and 5R(-) denote as the real operator. 
Thus, the MSD of b(n) can be computed as 



V b (n + l)=V b (n)-2 Fb (n)E^^Ab H (n)£ b (n 
+ ¥ 2 b {n)EU(^{n)i b (n 



where £/,(«) is calculated by 



Following these approximations 



€»(«) 



Mi>« 



b"b 



|d(n)| 



lim P,p(n + 1) = lim V w (n) , 
lim 2?b(n + 1) = lim V b (n) , 



(95) 

(96) 

(97) 
(98) 



are taken into Eq.(93) and Eq.(95). The normalised step-size parameters }iiu(n) and fi b (n) are 
bounded as 



< ji w (n) < 2 



h N 



Aw H (»)^(»; 

€£(n)€*(n) 

0< „ ( „ )<25e (^(^) 

Therefore, the optimal step-size parameters /i J and /i, ' can be formulated by 



opt 



opt 



V €g(n)« w (n) 
/ Ab H (H)g fc ( H ; 

V € b H («)&(n) 



(99) 
(100) 

(101) 
(102) 



7. Simulation results 



We implemented the ADSL transmission channel based on parameters as follows: the 
sampling rate f s = 2.208 MHz, the size of FFT N = 512, and the input signal power of 
-40dBm/Hz. The standard ADSL system parameters were shown in Table 1. The ADSL 
downstream starting at active tones 38 up to tone 255 that comprises 512 coefficients of 
channel impulse response. The signal to noise ratio gap of 9.8dB, the coding gain of 4.2dB 
and the noise margin of 6dB were chosen for all active tones. The additive white Gaussian 
noise (AWGN) with a power of — 140dBm/Hz and near-end cross talk (NEXT) from 24 
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Asymmetric Digital Subscriber Line (ADSL) Specifications 


Tapsofw(N„,) 32 


FFTsize (N) 


512 


Tapsofb(N, 7 ) 32 


Cyclic prefix (v) 


32 


Sampling rate (/„) 2.208 MHz 


Signal to noise ratio 


gap 9.8 dB 


Tone spacing 4.3125 KHz 


Noise margin 


6dB 


TX-DMT block (At) 400 


Coding gain 


4.2 dB 


TX sequence MxN 


Input power 


-40dBm/Hz 


Input impedance 100 fi 


AWGN power 


-140dBm/Hz 



Table 1 . The standard ADSL system for simulation. 

ADSL disturbers were included over the entire test channel. The optimal synchronisation 
delay (A) can be obtained from the proposed algorithm that was equal to 45. The ADSL 
downstream simulations with the carrier serving area (CSA) loop no. 1 was the representative 
of simulations with all 8 CSA loops as detailed in (Al-Dhahir & Cioffi, 1996). The CSA#1 loop 
is a 7700 ft, 26 gauge loop with 26 gauge bridged tap of length of 600 ft at 5900 ft. 

The initial parameters of the proposed AS-OSNALMS algorithm were w(0) = b(0) = 
W w (0) = f b (0) = [0.001 ■ ■ ■ 0] r and of NLMS algorithm were \i w = 0.15, }i b = 0.075. The 
NLMS algorithm was calculated with the fixed step-size for TEQ and TIR with the method 
as described in Section 4. Fig. 4 depicts the original simulated channel, SIR and TIR of the 
proposed AS-OSNALMS algorithm which compared with SIR of MMSE-UEC. It is noted that 
the comparable lengths of SIR and TIR of proposed algorithm are shorter than the original 
channel. This explains the channel-shortening capability of the proposed algorithm. Fig. 5 
illustrates the MSE curves of proposed AS-OSNALMS and NLMS algorithms. The MSE curve 
of proposed algorithm is shown to converge to the MMSE. Fig. 6 and Fig. 7 show the mean 
square deviation (MSD) on TEQ and TIR of proposed AS-OSNALMS and NLMS algorithms. 
The trajectories of }i w (n) and ,U{,(«) at the different of initial step-size \i Wa and /(;, are presented 
with the fixed at the adaptation parameters a w and «{, in Fig. 8 and Fig. 9 and with the 
different ct w and ccj, in Fig. 10 and Fig. 11. Comparing the proposed AS-OSNALMS algorithm 
with the fixed at the adaptation parameters, it has been shown that the proposed algorithms 
have faster initial convergence rate with the different setting of initial step-size and adaptation 
parameters. Their are shown to converge to their own equilibria. 



8. Conclusion 

In this chapter, we present the proposed adaptive step-size order statistic LMS-based TEQ 
and TIR for DMT-based systems. We introduce how to derive the updated tap-weight vector 
w(h) and b(n) as the solution of constrained optimisation to obtain a well-known NLMS 
algorithm, which an averaged order statistic scheme is replaced linear smoothing of the 
gradient estimation. We demonstrate the derivation of adaptive step-size mechanism for the 
proposed order statistic normalised averaged least mean square algorithm. The proposed 
algorithms for TEQ and TIR can adapt automatically the step-size parameters. The adaptation 
of MSE, MSD of TEQ and MSD of TIR curves of the proposed algorithms are shown to 
converge to the MMSE in the simulated channel. According to the simulation results, the 
proposed algorithms provide a good approach and are appeared to be robust in AWGN and 
NEXT channel as compared to the existing algorithm. 
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-original channel 

MMSE-UEC 

SIR of ASOSNALMS 

TIR of ASOSNALMS 




100 120 140 160 180 200 

discrete time (n) 



Fig. 4. Original channel, SIR of proposed ASOS-NALMS and TIR of AS-OSNALMS which compared 
with SIR of MMSE-UEC, when the samples of CSA loop are loop #1. Other parameters are 
]i lVa = 0.415, ]i ha = 0.095, k„ = 1.25 x 10~ 6 and ct b = 1.5 x 10~ 6 . 



AS-OSNALMS 
NLMS 




Fig. 5. Learning Curves of MSE of proposed AS-OSNALMS and NLMS algorithms for TEQ and TIR, 
when the samples of CSA loop are loop #1. Other parameters of AS-OSNALMS algorithm are 
fi u , o = 0.415, fi bo = 0.095, u m = 1.25 x 10~ 6 , x b = 1.5 X 10~ 6 and of NLMS alorithm are /!„, = 0.15, 
ji b = 0.075. 
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0.8 1 1.2 1.4 

discrete time (n) 



Fig. 6. Learning Curves of MSD V w {n) of proposed AS-OSNALMS and NLMS algorithms for TEQ, 
when the samples of CSA loop are loop #1. Other parameters of AS-OSNALMS algorithm are 
ji Wo = 0.415, }i b(1 = 0.095, a w = 1.25 x 10" 6 , a b = 1.5 x 10" 6 and of NLMS algorithm are /<„, = 0.15, 
fi b = 0.075. 




Fig. 7. Learning Curves of MSD V b (n) of proposed AS-OSNALMS and NLMS algorithms for TIR, when 
the samples of CSA loop are loop #1. Other parameters of AS-OSNALMS algorithm are }i u , = 0.415, 
]i ba = 0.095, a l0 = 1.25 x 10~ 6 , a h = 1.5 x 10~ 6 and of NLMS algorithm are y. w = 0.15, ]i b = 0.075. 
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Fig. 8. Trajectories of }i w of proposed AS-OSNALMS algorithm for TEQ using different 
setting of }i Wa and }i ho for TEQ and TIR with fixed at a w = 4.45 x 10~ 4 and a b = 1.75 x 10~ 4 , 
when the samples of CSA loop are loop #1. 
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Fig. 9. Trajectories of /.(;, of proposed AS-OSNALMS algorithm for TIR using different setting 



of ]iiu a and Hi, for TEQ and TIR with fixed at cc u 
the samples of CSA loop are loop #1. 
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Fig. 10. Trajectories of ]i w of proposed AS-OSNALMS algorithm for TEQ using different 
setting of \i Wa and \i^ a for TEQ and TIR with different at oc w = 4.45 x 10~ 4 and 
a w = 1.25 x 10~ 6 , when the samples of CSA loop are loop #1. 




■(i =0.0950, a =1.50x10 
- (i =0.0095. a =1 .75x10"' 



1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 
discrte time (n) 



Fig. 11. Trajectories of ]i\, of proposed AS-OSNALMS algorithm for TIR using different 
setting of pi Wa and y.f, a for TEQ and TIR with different at tXj, = 1.75 x 10~ 4 and 
#i, = 1.5 X 10~ 6 , when the samples of CSA loop are loop #1. 
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1. Introduction 

The modeling of oscillators and their dynamics has interested researchers in many fields such 
as those in physics, chemistry, engineering, and biology. The Hodgkin-Huxley (Hodgkin 
& Huxley, 1952) and Fitzhugh-Nagumo models (FitzHugh, 1961), which corresponds 
to the Bonhoffer van der Pol (BvP) equation, are well-known models of biological 
neurons. They have been described by differential equations, i.e., they are continuous-time 
relaxation oscillators. Discrete-time oscillators, e.g., one consisting of a recurrent neural 
network (Haschke & Steil, 2005) and another consisting of a spiking neuron model (Rulkov, 
2002), have been proposed. 

Synchronization observed in coupled oscillators has been established to be an important 
topic (Pikovsky et al., 2003; Waller & Kapral, 1984). Research on coupled oscillators 
has involved studies on pattern formation (Kapral, 1985; Oppo & Kapral, 1986), image 
segmentation (Shareef et al., 1999; Terman & Wang, 1995; Wang & Terman, 1995; 1997), 
and scene analysis (Wang, 2005). Of these, a locally excitatory globally inhibitory oscillator 
network (LEGION) (Wang & Terman, 1995), which is a continuous-time dynamical system, 
has been spotlighted as an ingenious image-segmentation system. A LEGION can segment 
an image and exhibit segmented images in a time series, i.e., it can spatially and temporally 
segment an image. We call such processing dynamic image segmentation. A LEGION consists 
of relaxation oscillators arranged in a two-dimensional (2D) grid and an inhibitor globally 
connected to all oscillators and it can segment images according to the synchronization of 
locally coupled oscillators. Image segmentation is the task of segmenting a given image so 
that homogeneous image blocks are disjoined; it is a fundamental technique in computer 
vision, e.g., object recognition for a computer-aided diagnosis system (Doi, 2007) in medical 
imaging. The problem with image segmentation is still serious, and various frameworks have 
been proposed (Pal & Pal, 1993; Suri et al., 2005) to solve this. 

We proposed a discrete-time oscillator model consisting of a neuron (Fujimoto et al., 2008), 
which was modified from a chaotic neuron model (Aihara, 1990; Aihara et al., 1990), 
coupled with an inhibitor. Despite discrete-time dynamics as well as the recurrent neural 
network (Haschke & Steil, 2005), a neuron in our oscillator can generate a similar oscillatory 
response formed by a periodic point to an oscillation as observed in a continuous-time 
relaxation oscillator model, e.g., the BvP equation. This is a key attribute in our idea. 
Moreover, we proposed a neuronal network system consisting of our neurons (discrete-time 
oscillators) arranged in a 2D grid and an inhibitor globally coupled to all neurons. As well as 
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a LEGION, our neuronal network system can work as a dynamic image-segmentation system 
according to the oscillatory responses of neurons. Our system provides much faster dynamic 
image segmentation than a LEGION on a digital computer because numerical integration is 
not required (Fujimoto et al., 2008). Another advantage of our system is that it simplifies 
the investigation of bifurcations of fixed points and periodic points due to the discrete-time 
dynamical system. A fixed point and a periodic point correspond to non-oscillatory and 
periodic oscillatory responses. Knowledge on the bifurcations of responses allows us to 
directly design appropriate system parameters to dynamically segment images. The assigned 
system parameters are made available by implementing our dynamic image-segmentation 
system into hardware such as field-programmable gate array devices (Fujimoto et al., 2011b). 
This article describes the derivation of a model reduced from our dynamic 
image-segmentation system that can simplify bifurcation analysis. We also explain our 
method of bifurcation analysis based on dynamical systems theory. Through analysis in 
reduced models with two or three neurons using our method of analysis, we find parameter 
regions where a fixed point or a periodic point exists. We also demonstrate that our 
dynamic image-segmentation system, whose system parameters were appropriately assigned 
according to the analyzed results, can work for images with two or three image regions. 
To demonstrate that segmentation is not limited to three in the system, we also present a 
successive algorithm for segmenting an image with an arbitrary number of image regions 
using our dynamic image-segmentation system. 

2. Discrete-time dynamic image-segmentation system 

2.1 Single neuronal system 

Figure 1(a) illustrates the architecture of a system consisting of a neuron (Fujimoto et al., 2008) 
and an inhibitor. Here, let us call it a single neuronal system. Our neuron model modified from 
a chaotic neuron model (Aihara, 1990; Aihara et al., 1990) has two internal state variables, x 
and y; z corresponds to the internal state variable of an inhibitor, in which x, y, z G R with R 
denoting the set of real numbers. Let the sum of internal state values in a neuron, i.e. x + y, 
be the activity level of a neuron. The dynamics of the single neuronal system is described by 
difference equations: 

x(t + l)=k f x(t)+d+W x -g(x(t)+y(t),e c )-W z -g(z(t),e z ) (la) 

y(t + l) = k r y(t) - a ■ g{x(t) + y(t),6 c ) + a (lb) 

z(t + l)=<p{g(g(x(t)+y(t),8 f ),8 d )-z(t)}. (lc) 

The t£Z denotes the discrete time where Z expresses the set of integers. g{-,-) is the output 
function of a neuron or an inhibitor and is described as 

^^ e) = i + eM-(u(t)-o )/£ y (2) 

Note that g(-, Q£) -where g{x(t) + y(t), 9r) is nested in Eq. (lc) is neither output function, but a 
function to find the firing of a neuron that corresponds to a high level of activity. Therefore, an 
inhibitor plays roles in detecting a fired neuron and suppressing the activity level of a neuron 
at the next discrete time. The kr, k r , and (p are coefficients corresponding to the gradient of x, 
y, and z. The d denotes an external direct-current (DC) input. The W x and a are self- feedback 
gains in a neuron, and W z is the coupling coefficient from an inhibitor to a neuron. The a is a 
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bias term in a neuron. The 6 C and Q z are threshold parameters in output functions of a neuron 
and an inhibitor, respectively. Also, Of and 8^ are threshold parameters to define the firing of 
a neuron and to detect a fired neuron, respectively. The £ is a parameter that determines the 
gradient of the sigmoid function (2) at u(t) = 0. 

When we set all the parameters to certain values, our neuron can generate a similar oscillatory 
response formed by a periodic point to an oscillation as observed in a continuous-time 
relaxation oscillator model. For instance, the time evolution of a generated response, in 
which this is a waveform, is shown in Fig. 1(b) for initial values, (x(0),j/(0),z(0)) = 
(32.108,-31.626,0.222), at k f = 0.5, d = 2, W x = 15, 8 C = 0, W 2 = 15, 6 Z = 0.5, k r = 0.89, 
ol = 4, a = 0.5, <p = 0.8, 6f = 15, 8 d = 0, and e = 0.1. To clarify the effect of the inhibitor, 
we have shown the activity level of the neuron and the internal state of the inhibitor on the 
vertical axis in this figure. The points marked with open circles "o" indicate the values of 
x + y and z at discrete time t. Although the response of a neuron or an inhibitor is formed 
by a series of points because of its discrete-time dynamics, we drew lines between temporally 
adjacent points as a visual aid. Therefore, our neuron coupled with an inhibitor is available as 
a discrete-time oscillator. 
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(b) Oscillatory response 



Fig. 1. Architecture of single neuronal system and generated oscillatory response 



2.2 Neuronal network system 

We have proposed a neuronal network system for dynamic image segmentation (Fujimoto 
et al., 2008). Figure 2(a) outlines the architecture of our system for a 2D image with P 
pixels. It is composed of our neurons that have as many pixels as in a given image and 
an inhibitor that is called a global inhibitor because it is connected with all neurons. All 
neurons are arranged in a 2D grid so that one corresponds to a pixel, and a neuron can have 
excitatory connections to its neighboring neurons. Here, we assumed that a neuron could 
connect to its four-neighboring ones. The formation of local connections between neighboring 
neurons is determined according to the value of DC input to each neuron. Note that, we 
can use our neuronal network system, in which neurons are arranged in a 3D grid so that 
one neuron corresponds to a voxel, which means a volumetric picture element, as a dynamic 
image-segmentation system for a 3D image. 

The architecture for the z'th neuron in a neuronal network system is illustrated in Fig. 2(b). 
The open and closed circles at the ends of the arrows correspond to excitatory and inhibitory 
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N : modified chaotic neuron 
GI: global inhibitor 



(a) Neuronal network system 
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(b) The ith neuron 

Fig. 2. Architecture of neuronal network system and a neuron 

couplings. A neuron can receive external inputs from neighboring ones connected to it. An 
external input from another neuron can induce in-phase synchronization in the responses 
of connected neurons. Note that the number of external inputs in Fig. 2(b) indexed by g{x^ + 
3/Jc/ 6c) is the same as that of the other neurons connected to the z'th neuron; moreover, when the 
DC-input value to the ith neuron is low, positive self-feedback vanishes and the neuron also 
has no connection to the others. W x / M; and W 2 in external inputs represent coupling weights; 
the other VV T /M,' and a are feedback gains, where M, denotes the number of connections to 
the ith neuron and neighboring neurons. What M, means will be explained later. 
The dynamics of our neuronal network system is described as 

x i (t + l) = k f x i (t)+d i + E ^g(**(t)+yjfc(*)A)-Wz-g(z(t),fe) (3a) 

keh, M ' 



y,(t + 1) = k r yi{t) - oc ■ g(xi(t) + yi{t), 9 C ) + a 



z(t+i) = <p\ g [ *Eg(x n (t)+y n (t),e f ),e d ) -z(t) 



(i = 1,2,..., P) 



(3b) 



(3c) 
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The g(-,-) was already defined in Eq. (2). The third term on the right hand side of Eq. (3a) 
denotes the zth neuron's self-feedback and external inputs from neighboring neurons, in 
which Lj represents an index set for neurons connected to the zth one. Therefore, the maximum 
number of elements in L, is five in the architecture in Fig. 2(a). The M,- expresses the number 
of elements in L,-. Note that, when the ith neuron has no connection to neighboring neurons 
including itself, i.e., M, = 0, we treat it as W x /Mj = because division by zero occurs. 
As seen in Eq. (3c), the dynamics of a global inhibitor is improved from that in Eq. (lc) so 
that it can detect one or more firing neurons; moreover, it suppresses the activity levels of all 
neurons via negative couplings described at the fourth term in the right hand side of Eq. (3a). 
Therefore, when we set all the parameter values in Eq. (3) to those described in Sec. 2.1, only 
neurons with self-feedback can generate oscillatory responses. 

2.3 Scheme of dynamic image segmentation 

There is an image segmentation scheme using our neuronal network system in Fig. 3. For 
simplicity, let us now consider a simple gray-level image with 3x3 pixels. The image contains 
two image regions consisting of the same gray-level pixels: the first is composed of the first 
and fourth pixels, and the second is made up of only the ninth pixel. 

Nine neurons are arranged in a 3 x 3 grid for the given image. The value of DC input, d,-, is 
associated with the gray level of the ith pixels. A neuron with a high DC-input value forms 
positive self-feedback and also connects to neighboring ones with similar DC-input values. 
Therefore, the red and blue neurons in this schematic have positive self-feedback connections 
and can generate oscillatory responses; the others corresponding to black pixels have no 
self-feedback and do not fire. Direct connection is formed between the red neurons because 
they correspond to pixels with the same gray levels, i.e., they have the same DC-input values. 
As seen from the red waveforms in Fig. 3, direct connection induces in-phase synchronization 
in the responses of coupled neurons. However, as seen from the red and blue waveforms, 
the responses of uncoupled neurons corresponding to pixels in different image regions are 
out of phase. This effect is produced by the global inhibitor that detects one or more firing 
neurons and suppresses the activity levels of all neurons with its own kindling. By assigning 
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Fig. 3. Scheme of dynamic image segmentation using our system 
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the /"th pixel value in the output image at time t to a high value corresponding to the white 
pixel only if Xj(t) + y;(f) > 8r, segmented images are output and are exhibited in a time 
series. As a result, the given image is spatially and temporally segmented, i.e., dynamic image 
segmentation is achieved. 

3. Analysis for parameter design 

3.1 Reduced model 

Our neuronal network model has complex dynamics and a variety of nonlinear phenomena 
such as synchronized neuron responses and bifurcations in these responses are therefore 
expected to occur in our system. From the viewpoint of dynamical systems theory, detailed 
analyses of the local and global bifurcations observed in our system would be interesting. 
However, we have only concentrated on analysis to design appropriate parameter values for 
dynamic image segmentation in this article, i.e., to find parameter regions where there are 
stable non-oscillatory or periodic oscillatory responses. 

First, we need to derive a reduced model to simplify bifurcation analysis. Let us consider 
a dynamic image-segmentation system for a P-pixel image with Q image regions, where 
generally Q -C P. A reduced model consists of a global inhibitor and Q neurons without 
direct coupling to the others as illustrated in Fig. 4. Here, we call it a Q-coupled system. 
A neuron in a Q-coupled system stands for neurons corresponding to all pixels in the same 
image region in our original neuronal system in Fig. 2(a). This reduced model is derived from 
three assumptions (Fujimoto et al., 2009b) in our dynamic image segmentation system for an 
image with Q image regions: 1) all pixel values in an identical image region are the same; 
viz., all neurons corresponding to pixels in an image region have the same DC-input values 
and are locally coupled with one another, 2) the responses of all neurons corresponding to 
pixels in an identical image region are synchronized in phase; this arises naturally from the 
first assumption, and 3) connections from the global inhibitor to the neurons are negligible 
because neurons corresponding to pixels with low gray-levels do not fire. 
A non-oscillatory response and a periodic oscillatory response correspond to a fixed point and 
a periodic point. Therefore, knowing about their bifurcations in a Q-coupled system allows 
us to directly design appropriate parameter values to dynamically segment any sized image 
with Q image regions. 




An input image with 
Q isolated image regions 



q>Vq) 



Fig. 4. Architecture of Q-coupled system and its correspondence to image with Q image 
regions 
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Now, let a;(f) = (x\{t),y\{t), . . . ,XQ(t),i/Q(t),z(t)) T e R y , where T denotes the transpose of 
a vector. The dynamics of the Q-coupled system is described by a V-dimensional discrete-time 
dynamical system where V = 2Q + 1 as 

x(t + l) = f(x(t)), (4) 

or equivalently an iterated map defined by 

/ :R y -^R v ;x^ f(x). (5) 

The nonlinear function, /, describes the dynamics of the Q-coupled system given by 






XQ 

vq 

V z / 



fkfXx + & x + W x ■ g(x-i + y x ,B c ) - W z ■ g(z,0 2 ) \ 
k r y\ -ct-g(xi +yi,9 c ) + a 



kfXQ + d Q + W x ■ g(x Q + y Q , B c ) - 
k r y Q - a ■ g(x Q + y Q ,8 c )+a 

g{ t^g(xn+yn,e f ),9 d \ -Z 



W z -g(z,8 z 



(6) 



where ; 



was defined in Eq. (2). 



3.2 Method of bifurcation analysis 

A non-oscillatory response observed in the Q-coupled system corresponds to a fixed point of 
/ in Eq. (5), and a periodic oscillatory response is formed by a periodic point of /. Therefore, 
we can find their local bifurcations for the change in a system parameter value using a method 
of analysis based on qualitative bifurcation theory for discrete-time dynamical systems. The 
results from analyzing bifurcation in a reduced model enabled us to design suitable parameter 
values in our neuronal network system for dynamic image segmentation. 
The following explains our method of analysis. Let us now consider a point, x* , satisfying 



fix* 



(7) 



This is called a fixed point of / in Eq. (5) and corresponds to a non-oscillatory response 
observed in the Q-coupled system. The characteristic equation of x* is defined as 



det(pE-Df{x*)) =0, 



(8) 



where E and Df(x*) correspond to the V x V identity matrix and the Jacobian matrix of / at 
x = x*. Moreover, the roots of Eq. (8), i.e., characteristic multipliers, are described as 



{jivm,---,Hv} = {}i t eC\ det fjd5-D/(as*))=0}, 



(9) 



where C denotes the set of complex numbers. When the values of all \m\s are neither unity 
nor zero, we say that x* is hyperbolic. Now, let us assume x* is a hyperbolic fixed point. Let 
U be the intersection of R and the direct sum of the generalized eigenspaces of Df(x* ) such 
that \m\ > l,Vf; U is called the unstable subspace of R y . Moreover, let H = Df(x*)\jj. The 
topological type of a hyperbolic fixed point is classified according to the value of dim U and 
the sign of det H (Kawakami, 1984). 
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A hyperbolic fixed point bifurcates when its stability is varied, or more correctly its topological 
type is changed, according to variations in a system parameter value; change in a topological 
type occurs when one or more characteristic multipliers are on the unit circle in the complex 
plane. There are generally three types of co-dimension-one bifurcations, i.e., tangent 
(saddle-node), period-doubling, and Neimark-Sacker bifurcations. D-type of branching 
(pitchfork bifurcation) can also appear as a degenerate case of tangent bifurcation in only a 
dynamical system that is symmetrical. Tangent bifurcation or D-type of branching appears if 
\i = +1, period-doubling bifurcation occurs when ji = — 1, and Neimark-Sacker bifurcation 
is generated when ]i = elf, where j = \/— 1 except for \i = ±1. 

A bifurcation point of x* is computed by solving simultaneous equations consisting of Eqs. (7) 
and (8) as the values of x* and a system parameter are unknown; we employed Newton's 
method for the numerical determination. The Jacobian matrix of the simultaneous equations 
used in Newton's method is derived from the first and second derivatives of /. Note that, in 
Eq. (7), a fixed point, x* , becomes an m-periodic point by replacing / with /'", which denotes 
the m-times iteration of /, where m is a natural number such that m > 2. We can define an 
m-periodic point and its bifurcations according to /"'; moreover, we can numerically compute 
the bifurcation points of an m-periodic point as well as those of a fixed point. 
As previously mentioned, we focused on bifurcation analysis to design suitable parameter 
values for our dynamic image segmentation system. Therefore, we will next illustrate 
parameter regions where there are stable fixed or stable periodic points in two-parameter 
bifurcation diagrams. 

3.3 Results of analysis 

We will now illustrate parameter regions where there are stable fixed or periodic points with 
our method of analyzing bifurcations. Knowing about the bifurcations allows us to directly 
set system-parameter values that yield successful results for dynamic image segmentation. 
We treated a single neuronal system and two- and three-coupled systems and set the system 
parameter values in Eqs. (2) and (6) except for k r , (p, and d,s to e = 0.1, kc = 0.5, W x = 15, 
9 C = 0, W z = 15, 9 Z = 0.5, a = 4, a = 0.5, Of = 15, and 6 d = 0. In the bifurcation diagrams 
that follow, we used symbols G™, I™, NS™, and D™ to denote tangent, period-doubling, 
and Neimark-Sacker bifurcations, and D-type of branching for an m-periodic point. The 
subscript series number I was appended to distinguish bifurcation sets of the same type for 
an m-periodic point. Note that these symbols indicate bifurcations of a fixed point if m = 1. 

3.3.1 Single neuronal system 

This is the reduced model of a dynamic image segmentation system for an image with only 
one image region. Its architecture is outlined in Fig. 1(a). It may seem that the analysis of 
bifurcations observed in this reduced model is meaningless for dynamic image segmentation. 
However, the existence of a fixed point in this model leads to considerable knowledge to 
devise an algorithm for dynamic image segmentation as will be explained later. 
We set d = 2 and used k r and (p as unfixed parameters to analyze bifurcation. As shown in 
Fig. 1(b), an oscillatory response was observed in this model with k r = 0.89 and <p = 0.8. 
Moreover, we found a stable fixed point, x* = (32.244, -23.333, 0.22222), at k r = 0.85 and cp = 
0.8. We investigated a parameter region where there was a stable fixed point and also found 
the genesis of the oscillatory response (Fujimoto et al., 2009b). 

Figure 5 shows a two-parameter bifurcation diagram on a fixed point in the (A: r ,<f)-plane. We 
found three Neimark-Sacker bifurcation sets and located the shaded parameter region where 



Discrete-Time Dynamic Image-Segmentation System 



413 



there was a stable fixed point. When we gradually changed the value of k r under ip = 0.8 so 
that the parameter point passed through the bifurcation line indexed by NS\ from the shaded 
region to the non-shaded region, the stable fixed point destabilized on the Neimark-Sacker 
bifurcation line. As a result, an oscillatory response was generated as seen in Fig. 6. In the 
numerical simulation, we set k r = 0.88975 and ip = 0.8 that correspond to the parameter 
point in the neighborhood at right of NS\ in Fig. 5; the initial values were set to a;(0) = 
(32.10, —31.58,0.2222), which is in the vicinity of the destabilized fixed point. That is, this 
figure gives the time evolution in the transient state that starts from the destabilized fixed 
point to generate an oscillatory response. Although we observed an oscillatory response in 
the other non-shaded region surrounded by NS2 and NS3, it is not suited to dynamic image 
segmentation because of its small amplitude and short period. 
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Fig. 5. Bifurcations of fixed point observed in single neuronal system 
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Fig. 6. Oscillatory response caused by Neimark-Sacker bifurcation of stable fixed point 



3.3.2 Two-coupled system 

This two-coupled system consists of a global inhibitor and two neurons without direct 
coupling to other neuron. This was derived as a reduced model of our scheme to dynamically 
segment an image with two image regions. 

Here, the unfixed parameters were set to &\ = dj = 2, which means that all pixel values in 
the two image regions are the same. Therefore, this system is symmetrical for the exchange of 
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(xi,y\) and (£2,1/2) ^ roln Eq. (6) a * ^1 = ^2- The A: r and <f were used as unfixed parameters in 
the analysis of bifurcation that is discussed below. 

First, we investigated the bifurcations of a fixed point in a symmetrical two-coupled 
system (Fujimoto et al., 2009b) where we observed a stable fixed point, x* = 
(32.244, 32.244, -23.333, -23.333, 0.222), at k r = 0.85 and <p = 0.8. The occurrence of a fixed 
point is adverse for dynamic image segmentation because only black images are output; this 
means dynamic image segmentation has failed. By analyzing bifurcation for the fixed point, 
we obtained the two-parameter bifurcation diagram in Fig. 5, i.e., this is the same as that for 
the results obtained for the fixed point in the single neuronal system. 

We observed two types of oscillatory responses formed by periodic points at k r = 0.89 and 
(p = 0.8. Figure 7 shows in-phase and out-of-phase oscillatory responses in which the blue 
and red points correspond to the responses of the first and second neurons. To understand 
their phases better, we also drew phase portraits. 

Figure 8(a) illustrates bifurcation sets of several in-phase periodic points, and the line marked 
NS\ at the bottom left corresponds to the Neimark-Sacker bifurcation set of the fixed point. 
As seen in the figure, we found the tangent bifurcations of in-phase periodic points. There is 
a stable in-phase m-periodic point in the shaded parameter region surrounded by G™ and G 2 " 
for m = 60, 61, . . . , 70. Therefore, in-phase periodic points could be observed in the shaded 
parameter regions in the right parameter regions of NS\. Note that in-phase periodic points 
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(b) Out-of-phase oscillatory response 



Fig. 7. Different types of oscillatory responses observed in symmetric two-coupled system at 
k r = 0.89 and <p = 0.8 

are inappropriate for dynamically segmenting an image with two image regions (Fujimoto 
et al., 2009b). 

Next, we investigated the bifurcations of out-of-phase periodic points on the 
(/c,-,0)-plane. (Musashi et al., 2009). As shown in Fig. 8(b), their tangent bifurcations 
and D-type of branchings were found. For example, there are stable out-of-phase m-periodic 
points in the shaded parameter region surrounded by Gf and Dj" for m = 30, 32, 34, 36 for 
the observed periodic points. Note that the overlapping parameter region indicates that 
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Fig. 8. Bifurcations of perodic points observed in symmetric two-coupled system 
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Fig. 9. Out-of-phase oscillatory response observed in asymmetric two-coupled system at 
k r = 0.85 and <p = 0.8 

out-of-phase periodic points coexist. The whole parameter region where there are stable 
out-of-phase periodic points is much wider than that of stable in-phase periodic points. 
This is favorable for dynamic image segmentation, because an in-phase periodic point is 
unsuitable and an out-of-phase periodic point is suitable. 

We set d\ ^ &i in the two-coupled system, and therefore, the symmetry for the exchange of 
(x\,y{) and (X2/2/2) m Eq. (6) is lost. This asymmetric two-coupled system corresponds to a 
situation where an input image contains two image regions of different colors. No symmetric 
periodic points occur in this system; however, we could observe the asymmetric out-of-phase 
periodic point shown in Fig. 9. Note that it is difficult to determine whether a periodic point is 
symmetric only from waveforms and phase portraits; however, this is not important because 
the feasibility of dynamic image segmentation is not dependent on whether there is symmetry 
or not but on the number of phases in a periodic point. 

Figure 10(a) shows bifurcation sets of out-of-phase periodic points observed at &\ = 2 and 
&2 = 1.9. Different from the symmetric system, D-type of branching never appeared due to 
the asymmetric system; instead, period-doubling bifurcations were found. Comparing the 
extent of all the shaded parameter regions in Figs. 8(b) and 10(a), the asymmetric system is 
as wide as the symmetric system. Moreover, we set d\ = 2 and (p = 0.8 and investigated 
their bifurcations on the (A: r ,d2)-pl an e as seen in Fig. 10(b). This indicates that there were 
stable out-of-phase periodic points even if the value of \d\ — &2 | was large; in other words, the 
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difference between the gray levels of the pixels in the two image regions is large. This is also 
favorable for a dynamic image segmentation system. 
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Fig. 10. Bifurcations of out-of-phase perodic points observed in asymmetric two-coupled 
system 



3.3.3 Three-coupled system 

This model is composed of a global inhibitor and three neurons without direct coupling to 
the others and was derived as a reduced model of our dynamic segmentation of an image 
containing three image regions. As well as the aforementioned reduced models, we drew 
several two-parameter bifurcation diagrams to find the parameter regions such that a stable 
fixed point or a stable m-periodic point existed. 
When we set ^1 = ^2 = ^3 = 2, the three-coupled system was symmetric for a 



circular exchange of 



for 



Ki+vyi+i), ' 



1,2,3 where the value of i + 1 returns 



to 1 if ! = 3. In this symmetric system, we found a stable fixed point, x* = 
(32.244,-29.167,32.244,-29.167,32.244,-29.167,0.222), at k r = 0.88 and f = 0.8. In 
the results we investigated, we found the bifurcation diagram on the fixed point on the 
(k r ,<p) -plane was the same as the one in Fig. 5. Moreover, as well as those in the symmetric 
two-coupled system, we could observe in-phase oscillatory responses in only the right hand 
side region of NS\. The waveform of an in-phase oscillatory response and its phase portraits 
are shown in Fig. 11(a), where the blue, red, and green points correspond to the responses of 
the first, second, and third neurons. The results suggest that the Neimark-Sacker bifurcation 
set, NS\, causes in-phase oscillatory responses to generate and these are similar to those of the 
symmetric two-coupled system (Fujimoto et al., 2009b; Musashi et al., 2009). Therefore, this 
implies that the global bifurcation structure of a fixed point and the generation of in-phase 
oscillatory responses are intrinsic properties of the symmetric Q-coupled system. 
We also observed several oscillatory responses in certain parameter regions. Figures 11(b) 
and 11(c) show a two-phase and a three-phase periodic points. For the following reasons, we 
only focused on the bifurcations of three-phase periodic points that were appropriate for 
dynamically segmenting an image with three image regions. 

Figure 13 shows bifurcation sets of three-phase periodic points observed in the symmetric 
system. Tangent, period-doubling, and Neimark-Sacker bifurcations were observed. The 
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(a) In-phase oscillatory response 




(b) Two-phase oscillatory response 
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(c) Three-phase oscillatory response 



Fig. 11. Different types of oscillatory responses observed in symmetric three-coupled system 
at d 1 =d 2 = d 3 = 2, k r = 0.89, and <p = 0.8 




Fig. 12. Three-phase oscillatory response observed in asymmetric three-coupled system at 
d x =2,d 2 = 1.9, d 3 = 1.8, k r = 0.89, and <p = 0.8 

respective periodic points are symmetrical for the aforementioned circular exchange. 
However, as seen in Fig. 13, we could find no D-type of branching in these investigations. 
There is a stable three-phase periodic point in each shaded parameter region. Compared 
with the extent of the entire shaded parameter region in Fig. 8(b), that of the three-phase 
periodic points is small; however, it is sufficient to design the parameters of our dynamic 
image segmentation system. 

Next, we set d\ 7^ d 2 7^ d 3 , i.e., this model is asymmetric. Although this three-coupled 
system loses symmetry, there is a three-phase periodic point in certain parameters as shown in 
Fig. 12. We investigated the bifurcations of several three-phase periodic points observed in the 
asymmetric system and drew two bifurcation diagrams. Figure 14(a) shows the bifurcation 
sets of three-phase periodic points on the (k r ,(p) -plane. Of course, we found no D-type of 
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Fig. 13. Bifurcations of three-phase periodic points observed in symmetric three-coupled 
system 
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(b) d-t =2,d 2 = 1.9, and <p = 0.8 



Fig. 14. Bifurcations of three-phase periodic points observed in asymmetric three-coupled 
system 

branching because of the asymmetric system. There is a stable three-phase periodic point in 
each parameter region shaded by a pattern. The shape and size of the whole shaded parameter 
region where there are three-phase periodic points are similar to those in Fig. 13. 
As seen in Fig. 14(b), we also computed the bifurcations of three-phase periodic points 
observed at d\ = 1, d.2 = 1.9, and cj) = 0.8 on the (k r , d3)-plane. As we can see from the figure, 
there are several stable three-phase periodic points even if the value of do, is set as small as 1.5. 
This suggests that our dynamic image-segmentation system can work for an image with three 
regions having different gray levels. 



4. Application to Dynamic Image Segmentation 

We demonstrated successful results for dynamic image segmentation carried out by our 
system with appropriate parameter values according to the results analyzed from the two- and 
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three-coupled systems. Our basic concept was that we assigned system parameters to certain 
values such those in-phase oscillatory responses, which are unsuitable for dynamic image 
segmentation. They do not appear but a multiphase periodic point with as many phases as 
image regions does occur. 

4.1 Image with two image regions 

Let us consider a dynamic image segmentation problem for the 8-bit gray-level image with 
256 x 256 pixels shown in Fig. 15(a). This is a slice from the X-ray CT images of the human 
head from the Visible Human Dataset (Ackerman, 1991). Using a thresholding method, we 
transformed the gray-level CT image into a binary image in preprocessing with Vz, p, = 
{0,255}, where p, denotes the zth pixel value. Here, the black and white correspond to 
and 255. The process image contains the two white image regions shown in Fig. 15(b). The 
upper region corresponds to teeth and the mandible bone, and the lower regions indicate the 
cervical spine. 

We need a neuronal network system to segment the binary image consisting of 256 x 256 
neurons and a global inhibitor. The DC-input value to the zth neuron, d,-, was set to 2.0 for 
neurons corresponding to pixels in the two white image regions based on d, = 2p,/255. 
Therefore, we can design system-parameter values according to the analyzed results for the 
symmetric two-coupled system in Figs. 5 and 8. 

Based on the information in the bifurcation diagrams, we set the two unfixed parameters to 
k r = 0.885 and <p = 0.8, which correspond to a parameter point in the left neighborhood of 
NS\ in Fig. 5, so that no in-phase oscillatory responses appear from any initial values but a 
fixed point or an out-of-phase 36-periodic point does occur. Note that, any of the out-of-phase 
periodic points in Fig. 8(b) are available for dynamic image segmentation, and the period 
of the periodic point used in dynamic image segmentation corresponds to the period each 
segmented image appeared in output images that were exhibited in a time series. 
The binarized image was input to our dynamic image segmentation system with 256 x 256 
neurons and a global inhibitor. According to an out-of-phase 36-periodic point, our system 
output images in the time series shown in Fig. 15(c), i.e., images were dynamically segmented 
successfully. Note that the output images sequentially appeared from the top-left to the 
bottom-right, and they also began to appear in each line from the left; moreover, output 
images corresponding to state variables in the transient state were removed. We confirmed 
from the series of output images that the period where each image region appeared was 36. 

4.2 Image with three image regions 

We considered the 8-bit gray-level image with 128 x 128 pixels shown in Fig. 16(a). It has three 
image regions: a ring shape, a rectangle, and a triangle. To simplify the problem, the color in 
each image region was made into a monotone in which the pixel values were 255, 242, and 230 
so that these values corresponded to d\ =2,d.2 = 1.9, and d$ = 1.8 according to d, = 2p;/255. 
To dynamically segment the target image, we needed a neuronal network system consisting 
of 128 x 128 neurons and a global inhibitor. The DC-input value to the zth neuron, d,-, was 
set to 2.0 for neurons corresponding to pixels in the ring shape, 1.9 for those in the rectangle, 
and 1.8 for those in the triangle. The neuronal network system with 128 x 128 neurons could 
be regarded as an asymmetric three-coupled system. Therefore, according to the analyzed 
results in Fig. 14, e.g., we set the unfixed parameter values to k r = 0.875 and cp = 0.8 such that 
a three-phase 25-periodic point occurred in the asymmetric three-coupled system. 
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(a) CT image 



(b) 
Binarization 














(c) Output images in time series 
Fig. 15. Results of dynamic image segmentation based on out-of-phase oscillatory response 

In trials for randomly given initial values, we achieved a successful result where three image 
regions appeared separately, as shown in Fig. 16(b). In addition, because the output images 
were generated according to a three-phase 25-periodic point observed in the asymmetric 
three-coupled system, we confirmed that the period where each image region appeared was 
25. Note that we removed output images corresponding to state variables in the transient 
state. Our neuronal network system could work for a simple gray-level image with three 
image regions. 

4.3 Image with many image regions 

To segment an image with an arbitrary number of image regions using our dynamic 
image-segmentation system in one process, it is necessary for a multiphase periodic oscillatory 
response with as many phases as image regions to appear. As far as our investigations 
were concerned, however, it was difficult to generate a multiphase periodic point with many 
phases. Therefore, we proposed an algorithm that successively and partially segments an 
image. 

Here, according to the previously mentioned results obtained from analysis, we considered 
a successive algorithm that partially segmented many image regions using two- and 
three-phase oscillatory responses. We let the gray-level image with five image regions in 
Fig. 17(a) be the target that should be segmented. To simplify the segmentation problem, we 
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(a) Target image 














(b) Output images in time series 
Fig. 16. Results of dynamic image segmentation based on three-phase oscillatory response 

assumed that all pixels in an identical image region would have the same gray levels. Based 
on the analyzed results for two- and three-coupled systems, we set the system parameters to 
certain values such that no in-phase oscillatory responses occurred but a fixed point or a two- 
or three-phase oscillatory responses appeared. 

We could obtain the three segmented images in Figs. 17(b)-(d) in the first step from the input 
image in Fig. 17(a) if a three-phase oscillatory response appeared. Note that the segmented 
images were extracted from output images in a time series by removing duplicate images and 
all black images. 

Each segmented image in Figs. 17(b)-(d) became an input image for our system in the next 
step. We obtained the two images in Figs. 17(e) and 17(f) in this step from the input image 
in Fig. 17(b) using a two-phase oscillatory response; as well as this process, we also obtained 
the two images in Figs. 17(g) and 17(h) from the input image in Fig. 17(c) according to the 
two-phase response; whereas we obtained no output images from the input image in Fig. 17(d) 
because the system to segment the image corresponded to a single neuronal system, and a 
fixed point always appeared under the system-parameter values we assigned. Therefore, the 
segmentation of the image in Fig. 17(d) was terminated in this step. 

The four images with only one image region in Figs. 17(e)-(h) are input images in the third 
step. As previously mentioned, we obtained no output images for an input image with only 
one image region. Therefore, our successive algorithm was terminated at this point in time. 
Thus, we could segment an image with an arbitrary number of image regions based on the 
successive algorithm. 
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Fig. 17. Schematic diagram of successive algorithm using our dynamic image-segmentation 
system 

5. Concluding remarks 

We introduced a discrete-time neuron model that could generate similar oscillatory responses 
formed by periodic points to oscillations observed in a continuous-time relaxation oscillator 
model. The scheme of dynamic image segmentation was illustrated using our neuronal 
network system that consisted of our neurons arranged in a 2D grid and a global inhibitor. 
Note that we suggested that a neuronal network system where neurons are arranged in a 3D 
grid can be applied to segmenting a 3D image. 

Images were dynamically segmented according to the responses of our system, and therefore, 
knowing about the bifurcations of the responses allowed us to directly set system-parameter 
values such that appropriate responses for dynamic image segmentation would appear. We 
derived reduced models that simplified our analysis of bifurcations observed in our neuronal 
network system and we found parameter regions where there was a non-oscillatory response 
or a periodic oscillatory response in the reduced models. According to the analyzed results, 
we set system parameters to appropriate values, and the designed system could work for two 
sample images with two or three image regions. Moreover, to segment an image with many 
image regions, we proposed a successive algorithm using our dynamic image-segmentation 
system. 

We encountered three main problems that should be solved to enable the practical use of our 
dynamic image-segmentation system: 

1. Development of a method that can form appropriate couplings between neurons for a 
textured image and a gray-level image containing gradation. 
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2. Development of a method that can give initial values to neurons and a global inhibitor so 
that an appropriate response will always appear. 

3. Development of a method or system that can provide fast processing using our system to 
segment a large-scale image and a 3D image within practical time limits. 

To solve the first problem, we proposed a dynamic image-segmentation system with a 
method of posterization (Zhao et al., 2003) used as preprocessing (Fujimoto et al., 2009a; 
2010). However, their method of posterization involves high computational cost and a 
large memory, we are considering a neuronal network system with plastic couplings as 
weight adaptation (Chen et al., 2000). We proposed a solution to the second problem with 
a method that avoids the appearance of non-oscillatory responses (Fujimoto et al., 2011a). 
However, toward an ultimate solution, we are investigating parameter regions such that no 
inappropriate responses appear through bifurcation analysis. An implementation to execute 
our dynamic image-segmentation system on a graphics processing unit is in progress as a 
means of rapid processing. 
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1. Introduction 

The problem of fault detection and diagnosis (FDD) in dynamic systems has received 
considerable attention in last decades due to the growing complexity of modern engineering 
systems and ever increasing demand for fault tolerance, cost efficiency, and reliability 
(Willsky, 1976; Basseville, 1988). Existing FDD approaches can be roughly divided into 
two major categories including model-based and knowledge-based approaches 
(Venkatasubramanian et al., 2003a; Venkatasubramanian et al., 2003b). Model-based 
approaches make use of the quantitative analytical model of a physical system. Knowledge- 
based approaches do not need full analytical modeling and allow one to use qualitative 
models based on the available information and knowledge of a physical system. Whenever 
the mathematical models describing the system are available, analytical model-based 
methods are preferred because they are more amenable to performance analysis. 
Generally, there are two steps in the procedure of model-based FDD. First, on the basis of 
the available observations and a mathematical model of the system, the state variable x and 
test statistics are required to be obtained. Then, based on the generated test statistics, it is 
required to decide on the potential occurrence of a fault. For linear and Gaussian systems, 
the Kalman filter (KF) is known to be optimal and employed for state estimation. The 
innovations from the KF are used as the test statistics, based on which hypothesis tests can 
be carried out for fault detection (Belcastro & Weinstein, 2002). In reality, however, the 
models representing the evolution of the system and the noise in observations typically 
exhibit complex nonlinearity and non-Gaussian distributions, thus precluding analytical 
solution. One popular strategy for estimating the state of such a system as a set of 
observations becomes available online is to use sequential Monte-Carlo (SMC) methods, also 
known as particle filters (PFs) (Doucet et al., 2001). These methods allow for a complete 
representation of the posterior probability distribution function (PDF) of the states by 
particles (Guo & Wang, 2004; Li & Kadirkamanathan, 2001). 

The aforementioned FDD strategies are single-model-based. However, a single-model-based 
FDD approach is not adequate to handle complex failure scenarios. One way to treat this 
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problem is the interacting multiple model (IMM) filter (Zhang & Li, 1998). For the IMM 
approach, the single-model-based filters running in parallel interact each other in a highly 
cost-effective fashion and thus lead to significantly improved performance. The initial 
estimate at the beginning of each cycle for each filter is a mixture of all most recent estimates 
from the single-model-based filters. It is this mixing that enables the IMM to effectively take 
into account the history of the modes (and, therefore, to yield a more fast and accurate 
estimate for the changed system states) without the exponentially growing requirements in 
computation and storage as required by the optimal estimator. The probability of each mode 
is calculated, which indicates clearly the mode in effect and the mode transition at each time. 
This is directly useful for the detection and diagnosis of system failures. In view of these, 
there is a strong hope that it will be an effective approach to FDD and thus has been 
extensively studied during the last decade, see (Zhang & Jiang, 2001; Yen &Ho, 2003; 
Tudoroiu & Khorasani, 2005; Rapoport & Oshman, 2007), and reference therein. 
A shortcoming of the IMM approach lies in that the mode declaration of the IMM filter may 
not reflect a true faulty situation because the model probability of the nominal model tends 
to become dominant especially when 1) the states and control inputs converge to the steady 
state at a nominal trim flight, or 2) a fault tolerant controller works well after the first failure. 
Besides, the IMM filter with the constant transition probability matrix has a problem 
diagnosing the second failure. To cope with the abovementioned problems, a new FDD 
technique is proposed using IMM filter and fuzzy logic for sensor and actuator failures. In 
this study, fuzzy logic is used to determinate the transition probability among the models 
not only to enhance the FDD performance after the first failure but also to diagnose the 
second one as fast and accurately as possible. 

On the other hand, fuel cell technology offers high efficiency and low emissions, and holds 
great promise for future power generation systems. Recent developments in polymer 
electrolyte membrane (PEM) technology have dramatically increased the power density of 
fuel cells, and made them viable for vehicular and portable power applications, as well as 
for stationary power plants. A typical fuel cell power system consists of numerous 
interconnected components, as presented comprehensively in the books (Blomen & 
Mugerwa, 1993), (Larminie & Dicks, 2000), (Pukrushpan et al. 2004b), and more concisely in 
the survey paper (Carette et al. 2001) and (Kakac et al. 2007). Faults in the fuel cell systems 
can occur in sensors, actuators, and the other components of the system and may lead to 
failure of the whole system (Hernandez et al. 2010). They can be modeled by the abrupt 
changes of components of the system. Typical faults of main concern in the fuel cell systems 
are sensor or actuator failures, which will degrade or even disable the control performance. 
In the last a few years, a variety of FDD approaches have been developed for various 
failures (Riascos et al., 2007; Escobet et al., 2009; Gebregergis et al. 2010). However, only 
simple failure scenarios, such as failure in sensor or actuator, are concerned therein. 
Moreover, upon FDD problem for the PEM fuel cell systems, there is little result so far by 
IMM approach. 

In this chapter, a self-contained framework to utilize IMM approach for FDD of PEM fuel 
cell systems is presented. As mentioned above, the constant transition probability matrix 
based IMM approach has problem in diagnosing the second failure, even though a fault 
tolerant controller works well after the first failure. Therefore, in our study, fuzzy logic is 
introduced to update the transition probability among multiple models, which makes the 
proposed FDD approach smooth and the possibility of false fault detection reduced. In 
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addition to the "total" (or "hard") actuator and/or sensor failures, "partial" (or "soft") faults 
are also considered. Compared with the existing results on FDD for fuel cell systems, more 
complex failure situations, including the total/ partial senor and actuator failures, are 
considered. Simulation results considering both single and simultaneous sensor and/ or 
actuator faults are given to illustrate the effectiveness of the proposed approach. 

2. IMM for fault detection and diagnosis revisited 

In this section, the details on generating the fault dynamics process using jump Markov 
linear hybrid dynamic models is first described. Then, the IMM estimation approach is 
developed for FDD. 

2.1 Jump Markov hybrid systems 

A stochastic hybrid system can be described as one with both continuous-valued base state 
and discrete-valued Structural/ parametric uncertainty. A typical example of such a system 
is one subject to failures since fault modes are structurally different from each other and 
from the normal (healthy) mode. An effective and natural estimation approach for such a 
system is the one based on IMMs, in which a bank of filters running in parallel at every time 
with jumps in mode modeled as transition between the assumed models. 
The IMM approach assumes that the state of the actual system at any time can be modeled 
accurately by the following jump Markov hybrid system: 

x(k + 1) = A(k,m(k + l))x(k) + B u (k,m(k + l))u(k) + B a (k,m(k + l))co(k r m(k + 1)) (1) 

x(0) S N(i ,P ) 

z(k) = C(k,m(k))x(k) + D u (k,m(k))u(k) + D u (k,m(k))u(k,m(k)) (2) 

with the system mode sequence assumed to be a first-order Markov chain with transition 
probabilities 

P{mj(k + 1) | m,-(fc)} = ^(k^Vm^ntj e S (3) 

and 

5>#) = 1,0 £ *#(*)* 1,» = 1 s (4) 

where x(k) is the state vector, z(k) is the mode-dependent measurement vector, and u(k) is 
the control input vector; a>(k) and u(k) are mutually independent discrete-time process and 
measurement noises with mean a>{k) and u(k) , and covariances Q(k) and R(k) ; P{] is the 
probability operator; m(k) is the discrete-value modal state (i.e., the index of the normal or 
fault mode in our FDD scenario) at time k, which denotes the mode in effect during the 
sampling period ending at t k ; n^ is the transition probability from mode m, to mode in •; 
the event that m is in effect at time k is denoted as mAk):={m(k) = mA . The mode set 
S = {m 1 ,m 2 ,...,fM s ) is the set of all possible system modes. 

The nonlinear system (l)-(2), known as a "jump linear system", can be used to model 
situations where the system behavior pattern undergoes sudden changes, such as system 
failures in this chapter and target maneuvering in (Li & Bar-Shalom, 1993). The FDD 
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problem in terms of the hybrid system may be stated as that of determining the current 
model state. That is, determining whether the normal or a faulty mode is in effect based on 
analyzing the sequence of noisy measurements. 

How to design the set of models to represent the possible system modes is a key issue in the 
application of the IMM approach, which is problem dependent. As pointed in (Li, 1996), this 
design should be done such that the models (approximately) represent or cover all possible 
system modes at any time. This is the model set design problem, which will be discussed in 
the next subsection. 

2.2 Model set design for IMM based FDD 

In the IMM method, assume that a set of N models has been set up to approximate the 
hybrid system (l)-(2) by the following N pairs of equations: 

x(k + 1) = AfiMk) + B uj (k)u(k) + B aj (k)co(k) (5) 

z(k) = Cj{k)x{k) + D uj (k)u(k) + D ul {k)v{k) (6) 

where N < s and subscript j denotes quantities pertaining to model m .■ e 5W ( SW is the set of 
all designed system models to represent the possible system modes in S . System matrices 
A ■, B u : , B , C , D , and D may be of different structures for different] . 
The model set design (i.e., the design of fault type, magnitude, and duration) is critical for 
IMM based FDD. Design of a good set of models requires a priori knowledge of the possible 
faults of the system. As pointed out in (Li & Bar-Shalom, 1996; Li, 2000), caution must be 
exercised in designing a model set. For example, there should be enough separation 
between models so that they are "identifiable" by the IMM estimator. This separation 
should exhibit itself well in the measurement residuals, especially between the filters based 
on the matched models and those on the mismatched ones. Otherwise, the IMM fault 
estimator will not be very selective in terms of correct FDD because it is the measurement 
residuals that have dominant effects on the model probability computation which in turn 
affect the correctness of FDD and the accuracy of overall state estimates. On the other hand, 
if the separation is too large, numerical problems may occur due to ill conditions in the set 
of model likelihood functions. A total actuator failures may be modeled by annihilating the 
appropriate column(s) of the control input matrix B„ and D u '■ 

x(k + 1) = A(k)x(k) + [B u (k) + M B] ]u(k) + BJk)a(k) (7) 

z(k) = C(k)x(k) + [D u (k) + M d] ]u{k) + D v (k)u(k) (8) 

That is, choose the matrix M B , with all zero elements except that the jth column is taken to 

be the negative of the jth column of B u . 

Alternatively, the jth actuator failure may be modeled by an additional process noise term 

e,(k): 

x(k + 1) = A(k)x(k) + B u (k)u(k) + B co (k)m(k) + £] (k) (9) 

z{k) = C{k)x{k) + D u {k)u{k) + D v {k)u{k) + e] {k) (10) 
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For total sensor failures, a similar idea can be followed. The failures can be modeled by 
annihilating the appropriate row(s) of the measurement matrix C described as 

z(k) = [C(k) + Lj]x(k) + D u (k)u(k) + D v {k)u(k) (11) 

or by an additional sensor noise term eJk) 

z(k) = C(k)x(k) + D u (k)u(k) + D v (k)u(k) + ej (k) (12) 

Partial actuator (or sensor) failures are modeled by multiplying the appropriate column (or 
row) of B u (or C) by a (scaling) factor of effectiveness. They can also be modeled by 
increasing the process noise covariance matrix Q or measurement noise covariance matrix R. 
Here we consider more complex failure situations, including total actuator and/ or sensor 
failures, partial actuator and/ or sensor failures, and simultaneous partial actuator and 
sensor failures. These situations require that the FDD algorithm be more responsive and 
robust. It is difficult for single-model-based approach to handle such complex failure 
scenarios. 

2.3 Procedures of IMM approach to FDD 

The following procedures should be performed in the application of the IMM estimation 

technique for fault detection and diagnosis: (i) filter reinitialization; (ii) model-conditional 

filtering; (iii) model probability updating; (iv) fault detection and diagnosis; (v) estimate 

fusion. 

The detailed steps for the IMM algorithm are described next (Zhang & Li, 1998; Mihaylova 

& Semerdjiev, 1999; Johnstone & Krishnamurthy, 2001). 

Step 1. Interaction and mixing of the estimates: filter reinitialization (interacting the 
estimates) obtained by mixing the estimates of all the filters from the previous time 
(this is accomplished under the assumption that a particular mode is in effect at the 
present time). 

1. Compute the predicted model probability from instant k to k+1: 

N 

Hj{k + l\k) = ^x ijMi {k) (13) 

1=1 

2. Compute the mixing probability: 

^ j (k) = M ij JU i (k)/ Mj (k + l\k) (14) 

3. Compute the mixing estimates and covariance: 



N 

I 

i'-l 



?(* I *) = £*<(* I *)%(*) (15) 



p°(k\k)=2{m\k)+ix°(k\k)-i i (k\k)ii°(k\k)-x i (k\k)] T } Mlll (k) (i6) 

1=1 
where the superscript denotes the initial value for the next step. 
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Step 2. Model-conditional filtering 

The filtering techniques such as (extended) Kalman filter, unscented Kalman filter, and 

particle filter can be applied for model-conditioning filtering. In this study, a linear Kalman 

filter is used as the individual filter of the IMM approach. 

Step 2.1: Prediction step 

1. Compute the predicted state and covariance from instant k to k+1: 

Xj(k + 1 1 k) = AjW&lik | k) + B uj (k)u(k) + B mj (k)m{k) (17) 

P ; (k + l\k) = A ; (k)P°(k\k)Aj(k) + B a; (k)Q J (k)Bl J (k) (18) 

2. Compute the measurement residual and covariance: 

r ] =z(k + l)-C ! (k + l)x 1 (k + l\k)-D u] (k)u(k)-D ul (k)U(k) (19) 

S, =Cj(k + l)Pj (k + 1 1 k)CJ(k + 1) + D vj (k)R(k)D T uj (k) (20) 

3. Compute the filter gain: 

K j =P j (k + l\k)C](k + l)Sj 1 (21) 

Step 2.2: Correction step 

Update the estimated state and covariance matrix: 

Xj{k + l\k + l) = Xj{k + l\k) + KjYj (22) 

Pj{k + 1 1 k + 1) = Pj(k + 1 1 k) - KjSjKj (23) 

Step 3. Updating the model probability 

The model probability is an important parameter for the system fault detection and 

diagnosis. For this, a likelihood function should be defined in advance, and then the model 

probability be updated based on the likelihood function. 

1. Compute the likelihood function: 



1 
LAk + 1) = . =exp 

2/rls ' 



\rJS? rj | (24) 



' 



2. Update the model probability: 



M } (k + 1) = — j/ l (25) 

Zl,M,(k +l|fc)L ; .(fc + l) 

Step 4. Fault detection and diagnosis 

1. Define the model probability vector Ji{k + 1) = \/u 1 (k + \),/J 2 {k + \),...,/u N (k +1)] . The 
maximum value of the model probability vector for FDD can be obtained as 
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^FDDmax= maX /'( fc + 1 ) ( 26 ) 

The index of the maximum value of the model probability vector component can be 
determined as 

; = find(/i FDDmax == fl(k + 1)) (27) 

2. Fault decision-FDD logic 

The mode probabilities provide an indication of mode in effect at the current sampling 
period. Hence, it is natural to be used as an indicator of a failure. According to the 
information provided by the model probability, both fault detection and diagnosis can be 
achieved. The fault decision can be determined by 

{> /Uj => Hi : Delare fault corresponding to jth mode 
TT XT i M. P 8 ) 

< n T => H 1 : No fault 

Or alternatively, 

» > /j t => H : Delare fault corresponding to jth mode 



max,«(fc + 1) < „' ->. H ■ No f au i t 

j7 V L 



(29) 



Step 5. Estimate fusion and combination that yields the overall state estimate as the 
probabilistically weighted sum of the updated state estimates of all the filters. The 
probability of a mode in effect plays a key role in determining the weights 
associated with the fusion of state estimates and co variances. The estimates and 
co variance matrices can be obtained as: 

N 

x(k + l\k + l) = '£x j {k + l\k + l)fi j {k + l) (30) 

P(fc + l|fc + l) = 

= Y j [p j (k\k)+{Z{k+i\k+i)-z j {k+i\k+i)){t{k+i\k+i)-x j {k+i\k+i)) T ] Mj {k+i) ( 3V> 

7=1 

It will be seen from Section 4 that the transition probability plays an important role in the 
IMM approach to FDD. In this study, the transition probability is adapted online through 
the Takagi-Sugeno fuzzy logic (Takagi & Sugeno, 1985). The overall framework of the 
proposed fuzzy logic based IMM FDD algorithm is illustrated in Fig. 1. 

It is worth noting that decision rule (28) or (29) provides not only fault detection but also the 
information of the type (sensor or actuator), location (which sensor or actuator), size (total 
failure or partial fault with the fault magnitude) and fault occurrence time, that is, 
simultaneous detection and diagnosis. For partial faults, the magnitude (size) can be 
determined by the probabilistically weighted sum of the fault magnitudes of the 
corresponding partial fault models. Another advantage of the IMM approach is that FDD is 
integrated with state estimation. The overall estimate provides the best state estimation of 
the system subject to failures. Furthermore, unlike other observer-based or Kalman filter 



432 



Discrete Time Systems 



based approaches, there is no extra computation for the fault decision because the mode 
probabilities are necessary in the IMM algorithm. Furthermore, the overall estimate is 
generated by the probabilistically weighted sum of estimates from the single-model-based 
filters. Therefore, it is better and more robust than any single-model-based estimate. This 
state estimate does not depend upon the correctness of fault detection and in fact, the 
accurate state estimation can facilitate the correct FDD. The detection threshold }{j is 
universal in the sense that it does not depend much on the particular problem at hand and a 
robust threshold can be determined easily. In other words, the FDD performance of the 
IMM approach varies little in most cases with respect to the choice of this threshold (Zhang 
& Li, 1998). On the other hand, the residual-based fault detection logic relies heavily on the 
threshold used, which is problem-relevant. Quite different detection thresholds have to be 
used for FDD problems of different systems and design of such a threshold is not trivial. 
Moreover, without comparing with the threshold, the value of the measurement residual 
itself does not provide directly meaningful detection and indication of the fault situations. 
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Fig. 1. Block diagram of the proposed fuzzy logic based IMM FDD approach 



3. Update of transition probability by fuzzy logic 

As aforementioned, the transition probability plays an important role in interacting and 
mixing the information of each individual filter. However, an assumption that the transition 
probability is constant over the total period of FDD can lead to some problems. Even if the 
fault tolerant control treats the first failure successfully, the unchanged transition probability 
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can mislead the FDD to intermittently declare a false failure alarm. This is because the fact 
that the normal mode before the first failure occurrence is not the normal mode any longer. 
The declared fault mode should be changed to a new normal mode after the first failure. On 
that account, the fuzzy-tuning algorithm of the transition probability is proposed in this 
study. 

The transition probability from any particular failure mode to the normal mode is generally 
set larger than others in order to prevent a false fault diagnosis. However, it may have a bad 
influence on performing correct fault diagnosis because the model probability of the healthy 
mode tends to increase again as the current failed system converges to the steady state by 
the fault tolerant control law even after a fault occurs. This problem can be overcome by 
adjusting the transition probability after the fault occurrence. For example, if the model 
probability of a certain failure mode remains larger than that of any other mode for an 
assigned time, the transition probability related to the corresponding failure mode should be 
increased. On the other hand, the transition probability related to the previous mode should 
be decreased to reflect the fact that the failed mode selected by the fault decision algorithm 
becomes currently dominant. In this work, the fuzzy-tuning algorithm is adopted to adjust 
the transition probabilities effectively. 

Now introduce a determination variable C, which decides whether or not the transition 
probabilities should be adjusted. First, the initial value of each mode's determination 
variable is set to zero. The increment of the determination variable can be obtained through 
the fuzzy logic with inputs composed of the model probabilities at every step. If the 
determination variable C; of a certain mode exceeds a predefined threshold value Ct> then 
the transition probabilities are adjusted, and the determination value of each mode is 
initialized. The overall process is illustrated in Fig. 2. 

3.1 Fuzzy input 

A fuzzy input for adjusting transition probabilities includes the model probabilities from the 
IMM filter. At each sampling time, the model probabilities of every individual filter are 
transmitted to the fuzzy system. In this work, the membership function is designed as in 
Fig. 3 for the fuzzy input variables "small," "medium," and "big" representing the relative 
size of the model probability. 

3.2 Fuzzy rule 

The T-S fuzzy model is used as the inference logic in this work. The T-S fuzzy rule can be 
represented as 

If ^f is A and ^ is B then Z=f(%, g ) (32) 

where A and B are fuzzy sets, and Z = /( % , £ ) is a non-fuzzy function. The fuzzy rule of 
adjusting transition probabilities is defined using the T-S model as follows 

If fii is small, then AC* = 

If jU: is medium, then AC'™ = 0.5 (33) 

If fij is big, then AC ; ° = 1 
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C,{k)=0 



Update DC,{k+l) using TS 
fuzzy logic 



C,(i+l)=C#)+AC,(t+l) 




Fig. 2. Flowchart of T-S fuzzy logic for adaptive model probability update 
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Fig. 3. Fuzzy membership function 
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3.3 Fuzzy output 

The output of the fuzzy system using the T-S model can be obtained by the weighted 
average using a membership degree in a particular fuzzy set as follows: 



AC,(k)- 



w'AC* + wJACf + wjACj 
w s ; + wf + w h 



(34) 



where w\, w"' , and if is the membership degree in the jth mode for group small, 
medium, and big, respectively. During the monitoring process, the determination variable 
of the jth mode is accumulated as 

ACj(k + 1) = Cj(k) + ACj(k + 1) (35) 

The designed fuzzy output surface of the T-S fuzzy interference system is shown in Fig. 4. 




0.4 0.6 

miu 

Fig. 4. Output surface of the fuzzy interference system 

Once the determination variable of a certain fault mode exceeds the threshold value C T , 
then all the elements of the transition probability matrix from the other modes to the 
corresponding fault mode are increased. 

3.4 Transition probability design 

The diagonal elements of the transition probability matrix can be designed as follows 
(Zhang & Li, 1998). 



7i j: =max<W;,l-- 



(36) 



where T , r ■ , and / are the sampling time, the expected sojourn time, and the predefined 
threshold of the transition probability, respectively. For example, the "normal-to-normal" 
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transition probability, n xl , can be obtained by 7i n = 1 - T / t 1 (here t 1 denotes the mean 
time between failures) since T is much smaller than r r in practice. The transition probability 
from the normal mode to a fault mode sums up to 1 - ;r xl . To which particular fault mode it 
jumps depends on the relative likelihood of the occurrence of the fault mode. While in 
reality mean sojourn time of total failures is the down time of the system, which is usually 
large and problem-dependent, to incorporate various fault modes into one sequence for a 
convenient comparison of different FDD approaches, the sojourn time of the total failures is 
assumed to be the same as that of the partial faults in this work. 

"Fault-to-fault" transitions are normally disallowed except in the case where there is sufficient 
prior knowledge to believe that partial faults can occur one after another. Hence, by using (36), 
the elements of the transition probability related to the current model can be defined by 

p„=l- — , p n = ^- (37) 

Vn ^ Vn N1 V / 

P/ =l- — , P f =l-P f (38) 

where p n and p , are the diagonal elements of the normal and failure mode, respectively, 
and p n and pr are off-diagonal elements to satisfy the constraint that all the row sum of the 
transition probability matrix should be equal to one. In addition, N is the total number of the 
assumed models, and r„ and t, are the expected sojourn times of the normal and failure 
mode, respectively. 

After a failure declaration by the fuzzy decision logic, the transition probability from the 
other modes to the corresponding failure model (say the mth mode) should be increased, 
whereas the transition probabilities related to the nonfailed model should be relatively 
decreased. For this purpose, the transition probability matrix of each mode is set as 
follows. 



Vn> 
Vf 
Pn' 

Vf 



= j =m 

= j *m 

= j and j ^ m (39) 

^ m and j = m 



0, otherwise 

4. PEM fuel cell description and modeling 

The fuel cell system studied in this work is shown in Fig. 5. It is assumed that the stack 
temperature is constant. This assumption is justified because the stack temperature changes 
relatively slowly, compared with the -100 ms transient dynamics included in the model to 
be developed. Additionally, it is also assumed that the temperature and humidity of the 
inlet reactant flows are perfectly controlled, e.g., by well designed humidity and cooling 
subsystems. It is further assume that the cathode and anode volumes of the multiple fuel 
cells are lumped as a single stack cathode and anode volumes. The anode supply and return 
manifold volumes are small, which allows us to lump these volumes to one "anode" 
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volume. We denote all the variables associated with the lumped anode volume with a 
subscript (an). The cathode supply manifold (sm) lumps all the volumes associated with 
pipes and connection between the compressor and the stack cathode (ca) flow field. The 
cathode return manifold (rm) represents the lumped volume of pipes downstream of the 
stack cathode. In this study, an expander is not included; however, we will consider this 
part in future models for FDD. It is assumed that the properties of the flow exiting a volume 
are the same as those of the gas inside the volume. Subscripts (cp) and (cm) denote variables 
associated with the compressor and compressor motor, respectively. 




Supply 

£> Manifold [ 

(SM) 



Compressor 
(CP) 



Cooler & 
Humidifier 



- Return Manifold 



/V. 



(pjvn 




Fig. 5. Simplified fuel cell reactant supply system 

The rotational dynamics and a flow map are used to model the compressor. The law of 
conservation of mass is used to track the gas species in each volume. The principle of mass 
conservation is applied to calculate the properties of the combined gas in the supply and 
return manifolds. The law of conservation of energy is applied to the air in the supply 
manifold to account for the effect of temperature variations. Under the assumptions of a 
perfect humidifier and air cooler, and the use of proportional control of the hydrogen valve, 
the only inputs to the model are the stack current, I s t, and the compressor motor voltage, v c ,„. 
The parameters used in the model are given in Table 1 (Pukrushpan et al., 2004a). The model 
is developed primarily based on physics. However, several phenomena are described in 
empirical equations. The models for the fuel cell stack, compressor, manifolds, air cooler and 
humidifier are presented in state-space model as specified by (40)-(41) with the relating 
matrices given in Table 1. 



x = A r x + B„ r u + b„ 



(40) 



z = C r x + D„ 



D„ 



(41) 
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where u = [v cm ,I st ] , and z = [W ,p sm ,v st ] , the stochastic noise or disturbance a> models 
the uncertainties caused by the linearization and measurement noises, etc. Note that the 
nominal operating point is chosen to be P„rf=40 kW and X02 =2, which correspond to 
nominal inputs of I s t=191 Amp and D C m=164 Volt. The state vector 
X = [m Oi ,m Hi ,m Ni ,0) cp ,f sm ,m sm ,m Oi ,m wan ,p rm f . In more details, the fuel cell system model 
developed above contains eight states. The compressor has one state: rotor speed. The 
supply manifold has two states: air mass and air pressure. The return manifold has one 
state: air pressure. The stack has four states: O2, and N2 masses in the cathode, and H2 and 
vapor masses in the anode. These states then determine the voltage output of the stack. 



Symbol 


Variable 


Value 


Ym,dry 


Membrane dry density 


0.002 kg/cnv* 


'Vi-m,dry 


Membrane dry equivalent weight 


1.1 kg/mol 


tm 


Membrane thickness 


0.01275 cm 


n 


Number of cells in stack 


381 


Aft 


Fuel cell active area 


280 cm? 


d c 


Compressor diameter 


0.2286 m 


lq> 


Compressor and motor inertia 


531025 kg.mz 


Van 


Anode volume 


0.005 m3 


V ca 


Cathode volume 


0.01 m 3 


V sm 


Supply manifold volume 


0.02 m3 


v rm 


Return manifold volume 


0.005 m3 


{—D,rm 


Return manifold throttle discharge coefficient 


0.0124 


f*-T,rm 


Return manifold throttle area 


0.002 m2 


K-smfiut 


Supply manifold outlet orifice constant 


0.362931025 kg/ (s.Pa) 


kccifiiit 


Cathode outlet orifice constant 


0.217731025 kg(s.Pa) 


h 


Motor electric constant 


0.0153 V/(rad/s) 


k t 


Motor torque constant 


0.0153 N-m/A 


K-cm 


Compressor Motor circuit resistance 


0.816 V 


item 


Compressor Motor efficiency 


98% 



Table 1. Model parameters for vehicle-size fuel cell system 

Three measurements are investigated: compressor air flow rate, z\=\N cv , supply manifold 
pressure, Z2=p S m , and fuel cell stack voltage, Zz=V s t ■ These signals are usually available 
because they are easy to measure and are useful for other purposes. For example, the 
compressor flow rate is typically measured for the internal feedback of the compressor. The 
stack voltage is monitored for diagnostics and fault detection purposes. Besides, the units of 
states and outputs are selected so that all variables have comparable magnitudes, and are as 
follows: mass in grams, pressure in bar, rotational speed in kRPM, mass flow rate in g/sec, 
power in kW, voltage in V, and current in A. 

In this study, the simultaneous actuator and sensor faults are considered. The fuel cell 
systems of interest considered here have two actuators and three sensors. Therefore, there 
are potentially only six modes, with the first mode being designated as the normal mode as 
(40)-(41) and the other five modes designated as the faulty modes associated with each of 
the faulty actuators or sensors. 
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" -6.30908 -10.9544 


83.74458 24.05866" 






-161.083 


51.52923 -18.0261 






-18.7858 -46.3136 


275.6592 158.3741 




A c = 


0- 
1.299576 2.969317 


-17.3506 193.9373 
0.3977 -38.7024 0.105748 






16.64244 38.02522 


5.066579-479.384 






-450.386 


142.2084 -80.9472 






2.02257 4.621237 


-51.2108 






"0 


5.066579 -116.446 0" 




C c = 





10 






12.96989 10.32532-0.56926 




B uc = 


" 3.94668 
-0.03159-0.00398 


0" 
-0.05242 


T 




-0.29656 













Table 2. Parameters for the linear fuel cell model in (40)-(41) 





A 1 = 

"0.1779 -0.0333 0.0047 -0.1284 0.0245 -0.02 
0.0012 0.0004 0.0002 -0.0169 0.0019 0.002 
-0.0401 0.0263 0.0038 -0.3963 0.0415 0.0663 
0.0444 0.0177 0.0079 -0.5741 0.0676 0.0806 
0.0036 0.0012 0.0006 -0.0517 0.0057 0.0059 
0.0408 0.01 0.0053 -3.9786 0.3265 0.0543 
-0.0004 -0.0003 -0.0001 0.0031 -0.0005 -0.0011 
0.0035 0.0011 0.0006 -0.0408 0.0048 0.0055 






"0.0451 -0.0145" 
0.004 
0.0878 0.0034 
0.3634 -0.0028 
0.0123 -0.0003 
0.1474 -0.0032 
-0.0007 0.0013 
0.0097-0.0003 






C 1 = 

"0 5.0666 -116.446 0" 
00 1000 
12.9699 10.3253-0.5693 








-0.2966 





Table 3. Parameters for the discretized model 

Actuator (or control surface) failures were modeled by multiplying the respective column of 
B h1 and D wl , by a factor between zero and one, where zero corresponds to a total (or 
complete) actuator failure or missing control surface and one to an unimpaired (normal) 
actuator/ control surface. Likewise for sensor failures, where the role of B ul and D ul is 
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replaced with C 1 . It was assumed that the damage does not affect the fuel cell system 

dynamic matrix A 1 , implying that the dynamics of the system are not changed. 

Let sampling period T = 1 s. Discretization of (40)-(41) yields the matrices for normal mode 

A x = e A ° r , B ul =(j T o e A ^dT)B c ,B col =(f o e A ^dT)B <x , C 1= C C , D ul ,= D ul , D ol ,= D ul , which 

are specified in Table 3. 

The fault modes in this work are more general and complex than those considered before, 
including total single sensor or actuator failures, partial single sensor or actuator failures, 
total and partial single sensor and/ or actuator failures, and simultaneous sensor and 
actuator failures. 

5. Results and discussion 

Scenario 1: Single total/partial actuator faulty mode 

First, in order to compare the performance between the conventional IMM and the proposed 
fuzzy logic based IMM approach, consider the simplest situation in which only a single total 
(or partial) sensor or actuator is running failure. Specifically, only partial failure for the 
actuator according to the second control input, i.e. stack current, I s t, is considered. The 
failure occurs after the 50th sampling period with failure amplitude of 50%. Two models 
consisting the normal mode and second actuator failure with amplitude of 50% are used for 
the IMM filter. The fault decision criterion in (29) is used with the threshold fi T = 2.5 . The 
transition matrix for the conventional IMM and the initial for the proposed approach are set 
as follows 



n = 



0.99 0.01" 
0.1 0.9 



The results of the FDD based on our proposed approach are compared with that of the 
conventional IMM filter. Fig. 6 (a) and (b) represent the model probabilities of the 2 models 
and the mode index according to (29) for the conventional IMM, respectively. From Fig. 6, it 
is obvious that the model probability related to the failure model does not keep a dominant 
value for the conventional IMM approach. On that account, momentary false failure mode is 
declared after the failure although the approach works well before the first failure occurs, 
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Fig. 6. The model probabilities and the mode index for the conventional IMM approach 
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just as shown in Fig. 6 (b). The performance of the proposed fuzzy logic based IMM 
approach is stable to hold a higher model probability than that of the conventional filter (cf . 
Fig. 7 (a)-(b)). This concludes that the improved IMM approach has better performance and, 
more importantly, reliability that the conventional IMM filter. 





Snmples 



:::: ::::>>■ 



Fig. 7. The model probabilities and the mode index for the proposed fuzzy logic based IMM 
approach 

Scenario 2: Single total/partial sensor/actuator faulty mode sequence 

Consider the situation in which only a single total (or partial) sensor or actuator failure is 
possible. Then there are a total of 4 possible model (one normal plus 3 failure models) for 
sensor failure and 3 possible models (one normal plus 2 failure models) for actuator failures. 
Similarly, there are 4 partial sensor failure models and 3 partial actuator failures models. 
Due to the space limitation, only the simulation results for the sensor failure case are 
presented herein. Let the pairs (zi, u n ), (z.% Usi), (23, u S 2), (Z4, u s i) designate the measurements 
and corresponding causes associated with the normal/ fault-free mode, and sensor fault for 
the first to the third sensor, respectively. Furthermore, let the pair (25, w s -3p) denote the 
measurement and corresponding causes associated with the partial fault for the third sensor. 
Consider the sequence of events designated by z=[zi, z% z\, 23, z\, 24, z\, Zs, z{\ and u=[ u„, u s \, 
U n , Usi, U n , w s 3, u„, u s 3 p , u„], where the first, second, third total sensor failures, and the partial 
third sensor failure occur at the beginning of the time horizon windows [31, 50], [81, 110], 
[141, 180], and [211, 250], respectively. Note that z\ corresponds to the normal mode. The 
faults persist for the duration of 20, 30, 40, and 40 samples, respectively. 

Let the initial model probability for both the conventional IMM and the fuzzy logic based 
IMM approach /j(0) = [0.2, 0.2, 0.2, 0.2, 0.2] T . The transition matrix for the conventional 
IMM and the initial one for the proposed approach are set as 



0.96 0.01 0.01 0.01 0.01 

0.1 0.9 

n= 0.1 0.9 

0.1 0.9 

0.1 0.9 

The mode indexes as a function of sampling period for the conventional IMM and the fuzzy 
logic based IMM approach are compared in Fig. 8. 
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Fig. 8. The mode index in the 2nd scenario for (a) the conventional IMM; and (b) the fuzzy 
logic based IMM 

Scenario 3: Simultaneous faulty modes sequence 

Let the pairs (zi, w„), {z% m s i), (Z3, u S 2), (Z4, u S 3), (Z5, u„i), (ze, u a i) stand for the measurements 
and corresponding causes associated with the normal mode, sensor fault for the first to the 
third sensor, and the actuator fault for the first and second actuator, respectively. 
Furthermore, let (z 7 , w„ ls2 ), (z 8 , w„ 2s 2), (zg, u ala i), (zio, w ols3 ), (zn, u ali i), (z 12/ ii s2s 3) designate the 
measurements and the inputs due to the presence of simultaneous double faulty modes 
caused by different combination of sensors and actuators, respectively. For simplicity and 
clarity, only sensor and actuator partial failures are considered herein. 

The initial model probabilities are fi(Q) = 1 / N , where N - 12 represents the number of 
modes; the threshold model probability /j t = 2.5 ; and the initial one for the proposed 
approach are set as 

abbbbbbbbbbb 
dcOOOOOOOOOO 
dOcOOOOOOOOO 
rfOOcOOOOOOOO 
dOOOcOOOOOOO 
dOOOOcOOOOOO 
dOOOOOcOOOOO 
dOOOOOOcOOOO 
dOOOOOOOcOOO 
dOOOOOOOOcOO 
dOOOOOOOOOcO 
dOOOOOOOOOOc 

where a = 19/20, b = 1 / 220 , c = 9 / 10 , and d = 1 / 10 . Two faulty sequences of events are 
considered. The first sequence is 1-2-3-4-6-12, for which the events occur at the beginning of 
the 1 st , 51 st , 101 st , 141 st , 201 st , 251 st sampling point, respectively. The second sequence is 1-3- 
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5-7-8-9-10-11-12, for which the events occur at the beginning of the I s ', 41 st , 71 st , 111 st , 141 st , 
181 st , 221 st , 251 st , 281 st sampling point, respectively. Note that Zi corresponds to the normal 
mode. Then, for the first case the faults persist for the duration of 40, 40, 60, 50, and 50 
samples within each window. For the second case the faults persist for 30, 40, 30, 40, 40, 30, 
30, and 20 samples, respectively. For space reason, only the performance and capabilities of 
the proposed approach are shown. The results for the two cases are shown in Fig. 9 and Fig. 
10, respectively. A quick view on the results, we may find that there is generally only one 
step of delay in detecting the presence of the faults. However, a more insight on both figures 
may reveal that at the beginning of the mode 4, it always turns out to be declared as mode 
10, while taking mode 8 for mode 12, and vice versa. This may be attributed to the similarity 
between the mode 4 and 10, 8 and 12. However, the results settled down quickly, only 5-6 
samples on average. 
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Fig. 9. The mode index in the 3rd scenario of sequence 1-2-3-4-6-12 
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Fig. 10. The mode index in the 3rd scenario of sequence 1-3-5-7-8-9-10-11-12 
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6. Conclusion and future work 

A self-contained framework to utilize IMM approach for fault detection and diagnosis for 
PEM fuel cell systems has been presented in this study. To overcome the shortcoming of the 
conventional IMM approach with constant transition matrix, a Takagi-Sugeno fuzzy model 
has been introduced to update the transition probability among multiple models, which 
makes the proposed FDD approach smooth and the possibility of false fault detection 
reduced. Comparing with the existing results on FDD for fuel cell systems , "partial" (or 
"soft") faults in addition to the "total" (or "hard") actuator and/or sensor failures have also 
been considered in this work. Simulation results for three different scenarios considering 
both single and simultaneous sensor and/ or actuator faults have been given to illustrate the 
effectiveness of the proposed approach. 

The scenarios considered correspond to representative symptoms in a PEM fuel cell system, 
and therefore the set of the considered models can't possibly cover all fault situations that 
may occur. Note that in case the fuel cell system undergoes a fault that it has not seen 
before, there is a possibility that the system might become unstable as a result of the IMM 
algorithm decision. It is indeed very difficult to formally and analytically characterize this, 
but based on our extensive simulation results presented, all the faulty can be detected 
precisely and timely. 

It is worth mentioning that the main objective of this work was to develop and present 
simulation results for the applicability and the effectiveness of the fuzzy logic based IMM 
approach for fault diagnosis of a PEM fuel cell system. The proposed approach can be 
readily extended to IMM-based fault-tolerant control and provides extremely useful 
information for system compensation or fault-tolerant control subsequent to the detection of 
a failure. This work is under investigation and will be reported in the near future. 
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1. Introduction 

1.1 Definitions and basic properties 

Discrete event systems (DES) constitute a specific subclass of discrete time systems whose 
dynamic behavior is governed by instantaneous changes of the system state that are triggered 
by the occurrence of asynchronous events. In particular, the characteristic feature of discrete 
event systems is that they are discrete in both their state space and in time. The modeling 
formalism of discrete event systems is suitable to represent man-made systems such as 
manufacturing systems, telecommunication systems, transportation systems and logistic 
systems (Caillaud et al. (2002); Delgado-Eckert (2009c); Dicesare & Zhou (1993); Kumar & 
Varaiya (1995)). Due to the steady increase in the complexity of such systems, analysis and 
control synthesis problems for discrete event systems received great attention in the last two 
decades leading to a broad variety of formal frameworks and solution methods (Baccelli et al. 
(1992); Cassandras & Lafortune (2006); Germundsson (1995); Iordache & Antsaklis (2006); 
Ramadge & Wonham (1989)). 

The literature suggests different modeling techniques for DES such as automata (Hopcroft 
& Ullman (1979)), petri-nets (Murata (1989)) or algebraic state space models (Delgado-Eckert 
(2009b); Germundsson (1995); Plantin et al. (1995); Reger & Schmidt (2004)). Herein, we focus 
on the latter modeling paradigm. In a fairly general setting, within this paradigm, the state 
space model can be obtained from an unstructured automaton representation of a DES by 
encoding the trajectories in the state space in an w-dimensional state vector x(k) G X" at each 
time instant k, whose entries can assume a finite number of different values out of a non-empty 
and finite set X. Then, the system dynamics follow 

F{x(k+l),x(k))=0, x(k)eX n 

where F marks an implicit scalar transition function F : X" x X" — > X, which relates x(k) at 
instant k with the possibly multiple successor states x(k + 1) in the instant k + 1. Clearly, in 
the case of multiple successor states the dynamics evolve in a non-deterministic manner. 
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In addition, it is possible to include control in the model by means of an m-dimensional control 
input u(k) 6 U m at time instant k. This control input is contained in a so called control set (or 
space) IT", where U is a finite set. The resulting system evolution is described by 

F(x(k + l),x(k),u(k)) = 0, x(k) e X n ,u(k) e U m 

In many cases, this implicit representation can be solved for the successor state x(k + 1), 
yielding the explicit form 

x{k+\)=f{x{k), U {k)) (1) 

or 

x(k + l)=f(x(k)) (2) 

when no controls are applied. As a consequence, the study of deterministic DES reduces to 
the study of a mapping f : X" — ¥ X n , or / : X" x U m — ¥ X" if we consider control inputs, 
where X and U are finite sets, X is assumed non-empty, and n, i»£N are natural numbers. 
Such a mapping / : X" — > X" is denoted as a time invariant discrete time finite dynamical system. 
Due to the finiteness of X it is readily observed that the trajectory x,f(x),f(f(x)), ... of any 
point x £ X" contains at most | X" | = | X| different points and therefore becomes either cyclic 
or converges to a single point y 6 X" with the property f(y) = y (i.e., a fixed point of /). The 
phase space of f is the directed graph (X", E, n : E — > X" x X") with node set X", arrow set E 
defined as E := {(x,y) G X" x X" \ f(x) = y} and vertex mapping 

7T : E -> X" x X" 
(x,y) h-» (x,y) 

The phase space consists of closed paths of different lengths that range from 1 (i.e. loops 
centered on fixed points) to |X"| (the closed path comprises all possible states), and directed 
trees that end each one at exactly one closed path. The nodes in the directed trees correspond 
to transient states of the system. In particular, if / is bijective , every point x 6 X" is contained 
in a closed path and the phase space is the union of disjoint closed paths. Conversely, if every 
point in the phase space is contained in a closed path, then / must be bijective. A closed path 
of length s in the phase space of / is called a cycle of length s. We refer to the total number of 
cycles and their lengths in the phase space of / as the cycle structure of f. 

Given a discrete time finite dynamical system / : X" — > X", we can find in the phase space 
the longest open path ending in a closed path. Let m £ No be the length of this path. It is easy 
to see, that for any s > m the (iterated) discrete time finite dynamical system f s : X" — ¥ X" 
has the following properties 

1. V X G X", f s (x) is a node contained in one closed path of the phase space. 

2. If T is the least common multiple of all the lengths of closed paths displayed in the phase 
space, then it holds 

ys+AT = jrs y A e N md jrs+i ^ jrs y ■ £ | ^ ^ j _ ^ 

We call T the period number of /. If T = 1, / is called a fixed point system. 

In order to study the dynamics of such a dynamical system mathematically, it is beneficial to 
add some mathematical structure to the set X so that one can make use of well established 
mathematical techniques. One approach that opens up a large tool box of algebraic and graph 
theoretical methods is to endow the set X with the algebraic structure of a finite field (Lidl & 



1 Note that for any map from a finite set into itself, surjectivity is equivalent to injectivity. 
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Niederreiter (1997)). While this step implies some limitations on the cardinality |X| of the set 
X, at the same time, it enormously simplifies the study of systems / : X" — > X" due to the 
fact that every component function /, : X" — > X can be shown to be a polynomial function of 
bounded degree in n variables (Lidl & Niederreiter (1997), Delgado-Eckert (2008)). In many 
applications, the occurrence of events and the encoding of states and possible state transitions 
are modeled over the Boolean finite field F2 containing only the elements and 1. 

1 .2 Control theoretic problems - analysis and controller synthesis 

Discrete event systems exhibit specific control theoretic properties and bring about different 
control theoretic problems that aim at ensuring desired system properties. This section 
reviews the relevant properties and formalizes their analysis and synthesis in terms of the 
formal framework introduced in the previous section. 

1.2.1 Discrete event systems analysis 

A classical topic is the investigation of reachability properties of a DES. Basically, the analysis 
of reachability seeks to determine if the dynamics of a DES permit trajectories between given 
system states. Specifically, it is frequently required to verify if a DES is nonblocking, that is, if it 
is always possible to reach certain pre-defined desirable system states. For example, regarding 
manufacturing systems, such desirable states could represent the completion of a production 
task. Formally it is desired to find out if a set of goal states Xg C X" can be reached from a 
start state x G X". 

In the case of autonomous DES without a control input as in (2), a DES with the dynamic 
equations x(k + 1) = f(x(k)) is denoted as reachable if it holds for all x G X that the set X g is 
reached after applying the mapping / for a finite number of times: 

Vxg x"ieNs.t f k (x) e x g . (3) 

Considering DES with a control input, reachability of a DES with respect to a goal set Xg holds 
if there exists a control input sequence that leads to a trajectory from each start state x G X" 
to a state x(k) G X g , whereby x(k) is determined according to (1): 

Vx G X n 3k G N and controls h(0), . . . ,u(k - 1) G U m s.t. x(k) G X g . (4) 

Moreover, if reachability of a controlled DES holds with respect to all possible goal sets X g C 

X", then the DES is simply denoted as reachable and if the number of steps required to reach 

Xg is bounded by / G N, then the DES is called /-reachable. 

An important related subject is the stability of DES that addresses the question if the dynamic 

system evolution will finally converge to a certain set of states ((Young & Garg, 1993)). 

Stability is particularly interesting in the context of failure-tolerant DES, where it is desired to 

finally ensure correct system behavior even after the occurrence of a failure. Formally, stability 

requires that trajectories from any start state x G X" finally lead to a goal set Xg without ever 

leaving X g again. 

Regarding autonomous DES without control, this condition is written as 

VxG X"3IeNs.t. Vfc> l,f k (x) G X g . (5) 

In addition, DES with control input require that 

Vf G X"3k G N and controls w(0), . . . , u(k - 1) G U m s.t.VZ > k x(l) G X g , (6) 



2 A well-known result states that X can be endowed with the structure of a finite field if and only if there 
is a prime number peN and a natural number m 6 N such that |X| = p'". 
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whereby k = 1 for all x G X g . 

It has to be noted that stability is a stronger condition than reachability both for autonomous 

DES and for DES with control inputs, that is, stability directly implies reachability in both 



In the previous section, it is discussed that the phase space of a DES consists of closed paths 
- so-called cycles - and directed trees that lead to exactly one closed path. In this context, the 
DES analysis is interested in inherent structural properties of autonomous DES. For instance, 
it is sought to determine cyclic or fixed-point behavior along with system states that belong 
to cycles or that lead to a fixed point ((Delgado-Eckert, 2009b; Plantin et al., 1995; Reger & 
Schmidt, 2004)). In addition, it is desired to determine the depth of directed trees and the 
states that belong to trees in the phase space of DES. A classical application, where cyclic 
behavior is required, is the design of feedback shift registers that serve as counter circuits in 
logical devices ((Gill, 1966; 1969)). 

1 .2.2 Controller synthesis for discrete event systems 

Generally, the control synthesis for discrete event systems is concerned with the design of a 
controller that influences the DES behavior in order to allow certain trajectories or to achieve 
pre-specified structural properties under control. In the setting of DES, the control is applied 
by disabling or enforcing the occurrence of system events that are encoded by the control 
inputs of the DES description in (1). On the one hand, the control law can be realized as a 
feedforward controller that supplies an appropriate control input sequence u(0), u(\ ),..., in 
order to meet the specified DES behavior. Such feedforward control is for example required 
for reaching a goal set Xg as in (4) and (6). On the other hand, the control law can be stated 
in the form of a feedback controller that is realized as a function g : X n — > U m . This function 
maps the current state x G X" to the current control input g(x) and is computed such that the 
closed-loop system 

h : X" ->■ X" 
* >-► f(x,g(x)) 

satisfies desired structural properties. In this context, the assignment of a pre-determined 
cyclic behavior of a given DES are of particular interest for this chapter. 

1 .3 Applicability of existing methods 

The control literature offers a great variety of approaches and tools for the system analysis 
and the controller synthesis for continuous and discrete time dynamical systems that are 
represented in the form 

x(t) = f(x{t),u(t)) or x(k + l)=f(x(k),u{k)), 

whereby usually x(t) G W, u(t) G R m , and x(k) G W, u(k) G R m , respectively. 
Unfortunately, traditional approaches to analyzing continuous and discrete time dynamical 
systems and to synthesizing controllers may fail when dealing with new modeling paradigms 
such as the use of the finite field F2 for DES as proposed in Section 1.1. From a mathematical 
point of view, one of the major difficulties is the fact that finite fields are not algebraically 
closed. Also non-linearity in the functions involved places a major burden for the system 
analysis and controller synthesis. In general, despite the simple polynomial shape of the 
transition function / (see above), calculations may be computationally intractable. For 
instance, determining the reachability set ((Le Borgne et al., 1991)) involves solving a certain 
set of algebraic equations, which is known to be an NP-hard problem ((Smale, 1998)). 
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Consequently, one of the main challenges in the field of discrete event systems is the 
development of appropriate mathematical techniques. To this end, researchers are confronted 
with the problem of finding new mathematical indicators that characterize the dynamic 
properties of a discrete system. Moreover, it is pertinent to establish to what extent such 
indicators can be used to solve the analysis and control problems described in Section 1.2. 
In addition, the development of efficient algorithms for the system analysis and controller 
synthesis are of great interest. 

To illustrate recent achievements, this chapter presents the control theoretic study of linear 
modular systems in Section 2, on the one hand, and, on the other hand, of a class of nonlinear 
control systems over the Boolean finite field F2, namely, Boolean monomial control systems in 
Section 3, (first introduced by Delgado-Eckert (2009b)). 

2. Analysis and control of linear modular systems 3 

2.1 State space decomposition 

In this section, linear modular systems (LMS) over the finite field F2 shall be in the focus. Such 
systems are given by a linear recurrence 

x(k + l)=Ax{k), fceN , (7) 

where A g Fj x " is the so-called system matrix. As usual in systems theory, it is our objective 
to track back dynamic properties of the system to the properties of the respective system 
matrix. To this end, we first recall some concepts from linear algebra that we need so as to 
relate the cycle structure of the system to properties of the system matrix. 

2.1.1 Invariant polynomials and elementary divisor polynomials 

A polynomial matrix P(X) is a matrix whose entries are polynomials in A. Whenever 
the inverse of a polynomial matrix again is a polynomial matrix then this matrix is called 
unimodular. These matrices are just the matrices that show constant non-zero determinant. In 
the following, F denotes a field. 

Lemma 1. Let A e F" x " be arbitrary. There exist unimodular polynomial matrices !i(A),V(A) e 
F[A]" X " such that 

U(A)(XI - A)V(X) = S(A) (8) 

with 

f Cl (X) ■■■ ^ 

c 2 (A) : 

: '•■ 

\ ■■■ c„(A)/ 

in which c,-(A) £ F[A] are monic polynomials with the property c, +1 1 c,-, i = 1, . . .,n — 1. 

Remark 2. The diagonal matrix S(A) is the Smith canonical form of AI — A which, of course, exists 
for any non-square polynomial matrix, not only in case of the characteristic matrix XI — A. However, 
for A not in the spectrum of A the rank of XI — A is always full and, thus, for any non-eigenvalue X 
we have q(A) 7^ 0. 

Definition 3. Let A £ F " xn be arbitrary and S(X) G F[A] nxn the Smith canonical form associated 
to the characteristic matrix XI — A. The monic polynomials c; ( A), i = 1, . . . , n, generating S (A) are 
called invariant polynomials of A. 



S(A) 



(9) 



3 Some of the material presented in this section has heen previously published in (Reger & Schmidt, 
2004). 



452 



Discrete Time Systems 



It is a well-known fact that two square matrices are similar if and only if they have the same 
Smith canonical form ((Wolovich, 1974)). That is, these invariant polynomials capture the 
coordinate independent properties of the system. Moreover, the product of all invariant 
polynomials results in the characteristic polynomial cp^(A) = det(A I — A) = Cj(A) ■ ■ ■ c„(A) 
and the largest degree polynomial Ci(A) in S(A) is the minimal polynomial mp^(A) of A, 
which is the polynomial of least degree such that mp A (A) = 0. The invariant polynomials 
can be factored into irreducible factors. 

Definition 4. A non-constant polynomial p e F[A] ;'s called irreducible over the field F if whenever 
p(A) = g(X)h(X) in F[A] then either g(A) or /z(A) is a constant. 

In view of irreducibility, Gaufi' fundamental theorem of algebra can be rephrased so as to 
obtain the unique factorization theorem. 

Theorem 5. Any polynomial p G F[A] can be written in the form 



'Vi 



with a G F, e\, . . . ,e^ G N, and polynomials p\, 
is unique except for the ordering of the factors. 



■ ■ Vk k (10) 

, Pk £ F[A] irreducible over ~¥ . The factorization 



Definition 6. Let A g ]F" xn be arbitrary and c,- = Vi\" 'Vin e V[X\, i = l,...,h, 
the corresponding h non-unity invariant polynomials in unique factorization with N{ factors. The 
N = YJi=\ Nj' monk factor polynomials p-'-, i = 1, . . . , ii and j = 1, . . . , N,-, are called elementary 
divisor polynomials of A. 

In order to precise our statements the following definition is in order: 
Definition 7. Let pc: = A rf + E/=n a > ^' G ^W ^ e mon ^ c - Then the (d x d)-matrix 

\ 



C 









\ — flg — «1 — «2 



(11) 



-«d-2 -H-\' 



is called the companion matrix associated to p c . 

Based on Definition 7, it is now possible to define the rational canonical form of a given matrix. 

Theorem 8. Let A G F " x " be arbitrary and pM its N elementary divisor polynomials, as introduced 
in Definition 6. There exists an invertible matrix T such that 

^rat = T- 1 AT = diag(C 1 ,...,C N ) (12) 

Cn are the companion matrices associated to the N elementary divisor polynomials of 



where C\, 
A. 



Remark 9. Except for the ordering of the companion matrices the matrix A ra t is unique. Furthermore, 
the number N is maximal in the sense that there is no other matrix similar to A that comprises more 
than N companion matrices. 
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2.1 .2 Cycle structure 

As pointed out in the introductory section, the phase space of any discrete system may be 
decomposed into closed paths (cycles) and paths that terminate in some cycle. For ease of 
notation, let N% denote the number of different-length cycles in a discrete system. Moreover, 
let the expression v[t] denote v cycles of length r. For this notation it clearly holds v,-[t] + 
v j[ T ] = ( v i + "iOM- Then the formal sum (cycle sum) 

£ = v\ [ti] + v 2 [t 2 ] + . . . + v Nl [t Ne ] (13) 

is used to represent the entire cycle structure of a discrete system that has a total of V; cycles 
of length T; for i = 1, . . . , N-%. The cycle structure is naturally linked to the notion of a periodic 
state, which shall be introduced for the particular case of linear modular systems. 

Definition 10. Let x G F 2 " be a non-zero state of the LMS in (7). The period of x is the least natural 
number t such that x = Ax. The period of the zero state x = is t = 1. 

Without loss of generality, let the LMS in (7) be given in the elementary divisor version of the 
rational canonical form 4 (see Theorem 8). Hence, 

x(fc+l) = diag(C 1/ ...,C N )x(fc). (14) 

The representation reveals the decomposition of (7) into N decoupled underlying subsystems, 
Xj(k + 1) = C, X{(k), associated to the companion matrices Q with respect to each elementary 
divisor polynomial of A. By combinatorial superposition of the periodic states of the 
subsystems it is clear that the periods of the states in the composite system follow from the 
least common multiple of the state periods in the subsystems. Therefore, for the examination 
of the cycle structure, it is sufficient to consider the cycle structure of a system 

x(k + l) = Cx(k). (15) 

In this representation, C G Fj is a companion matrix whose polynomial p^ G 1*2 [A] is a 
power of a monk polynomial that is irreducible over F2, whereby either p^ (0) 7^ or Pq = A 
((Reger, 2004)). It is now possible to relate the cyclic properties of the matrix C to the cyclic 
properties of the polynomial pc- 

Theorem 11. Let a linear modular system x(k + 1) = Cx(k) be given by a companion matrix 
C G Fj and its corresponding d-th degree polynomial pc = (Pirr,c) e > where Pm,c £ ^[A] is 
an irreducible polynomial over F2 of degree 5 such that d = e5. Then the following statements hold: 

1. ifpirr,c(0) 7^ 0, then the phase space of the system has the cycle sum 

E = 1[1] + l — [ Tl ] + ■ ■ ■ + —^[Tj] + ...+ ^ [re] . (16) 

r l l j L e 

In the above equation, the periods t,, j = 1, . . . ,e are computed as Xi = 2 It, whereby r represents 
the period of the irreducible polynomial p„ Xi c which is defined as the least positive natural number 
for which p(A) divides A T — 1. In addition, L, j = 1, . . . , e, is the least integer such that Ti > j. 

2. If Pi rl/ c = A d , then the phase space forms a tree with d levels, whereby each level I = 1, . . . , d 
comprises 2 ,_1 states, each non-zero state in level I — 1 is associated to 2 states in level I, and the 
zero state has one state in level 1. 



Otherwise, we may always transform x = T x such that in new coordinates it will he. 
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Proof. Part 1. is proved in [Theorem 4.5 in (Reger, 2004)] and part 2. is proved in [Theorem 4.9 
in (Reger, 2004)]. ■ 

Equipped with this basic result, it is now possible to describe the structure of the state space 
of an LMS in rational canonical form (14). Without loss of generality, it is assumed that the 
first c companion matrices are cyclic with the cycle sums Ej, . . . , E c , whereas the remaining 
companion matrices are nilpotent. Using the multiplication of cycle terms as defined by 



H% 



1cm (t,, t. 



- [km(T,-, Tj)}= VfVj gcd(T;, Tj) [lcm(T;, Tj)], 



d-H } levels, that 
For the detailed 



the cycle structure E of the overall LMS is given by the multiplication of the cycle sums of the 
cyclic companion matrices 

E = Ej E2 ■ ■ ■ E c . 

Finally, the nilpotent part of the overall LMS forms a tree with max{d c+ i, . . . 

is, the length of the longest open path of the LMS is l = max{d c+ i, . . . , d^} 

structure of the resulting tree the reader is referred to Section 4.2.2.2 in (Reger, 2004). 

The following example illustrates the cycle sum evaluation for an LMS with the system matrix 

A G F| an d its corresponding Smith canonical form S(A) £ F2[A] 5x5 that is computed as 

in [p. 268 ff. in (Booth, 1967)], [p. 222 ff. in (Gill, 1969)]. 

/I 1 1\ 
'llOOl 

00101 , S(A) 

00001 

Vi 1/ 



r 


+ A + 1)(A + 1) 2 





0\ 





A+l 000 










100 










1 


V 








1/ 



Here the only non-constant invariant polynomials of A are 

Cj(A) = (A 2 + A + 1)(A + 1) 2 , c 2 {\) 



A + l 



as indicated by the Smith canonical form. Thus, A has the elementary divisor polynomials 

p Cl (A)=A 2 + A + l, PC2 (A) = (A + 1) 2 , p C3 (A)=A + l. 

Since none of the elementary divisor polynomials is of the form A for some integer h, the 
system matrix A is cyclic. The corresponding base polynomial degrees are &\ = 2, 5 2 = 1 and 
<% = 1, respectively. Consequently, the corresponding rational canonical form /l ra t = T A T _1 
together with its transformation matrix T reads 



1 0\ 



/0 1 
1 1 



A ra t = diag(Ci, C 2 ,C 3 ) 



000 
00010 
00100 

Vo 1/ 



,T 



(\ 1 1\ 

'OOOll* 

10 10 1 

00111 

Vo 1 1 1/ 



,T~ 



/l 1 0\ 
1110 1' 

01010 
11110 

Vi 1 1 0/ 



5 A matrix A is called nilpotent when there is a natural number n £ N such that A" = 0. 

6 A simple method for obtaining the transformation matrix T can be found in Appendix B of (Reger, 



2004). 



-1 (1) = 


= 3 


,f= 


= 1 


T (2). 


=2 


(3) 


= 1 
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In view of Theorem 11, the corresponding periods are 

?vc 1 ( a ) = a2 + a + 1 I a3 + 1 => 

Pirr,C 2 ( A ) = A + 1 =*• 

(Pirr,C 2 ( A )) 2 = ( A + l) 2 = A2 + l => 

Pirr,C 3 ( A ) = A + ! => 

Thus, the associated cycle sums are 

E 1 = l[l] + [3], E 2 = 2[l] + [2], E 3 = 2[l]. 

The superposition of these cycle sums yields the cycle sum of the overall LMS 

E = E X E 2 E 3 = (1[1] + 1[3]) (2[1] + 1[2]) (2[1]) = (2[1] + 1[2] + 2[3] + 1[6]) (2[1]) = 
= 4[l]+2[2]+4[3]+2[6]. 

Therefore, the LMS represented by the system matrix A comprises 4 cycles of length 1, 2 cycles 
of length 2, 4 cycles of length 3 and 2 cycles of length 6. 

2.2 Reachability and stability 

In this section, the DES properties of reachability and stability as introduced in Section 1.2 
are investigated. The DES analysis for both properties is first performed for systems with no 
controls in Subsection 2.2.1. In this case, we can prove necessary and sufficient conditions for 
reachability and stability for general (not necessarily linear) deterministic DES / : X" — > X", 
without even requiring an algebraic structure imposed on the set X. However, to achieve 
equivalent results in the case of DES with controls, we need to endow the set X with the 
structure of a finite field and assume that the mapping / : X" x U m — > X" is linear. This is 
presented in Subsection 2.2.2. 

2.2.1 Reachability and stability for discrete event systems with no controls 

The reachability analysis for DES with no controls requires the verification of (3). As 
mentioned in Section 1.1, any state x G X either belongs to a unique cycle or to a tree that 
is rooted at a unique cycle. In the first case, it is necessary and sufficient for reachability from 
x that there is at least one state x G X g that belongs to the same cycle. Denoting the cycle 
length as T, it follows that/ (x) = x G Xg for some < k < r. In the latter case, it is sufficient 
that at least one state x 6 X g is located on the cycle with length t where the tree is rooted. 
With the length l of the longest open path and the root x t of the tree, it holds that x t = f (x) 
with < I < l and x = f k {x T ) for some < k < t. Hence, f ,+k x = x £ X g . Together, it 
turns out that reachability for a DES without controls can be formulated as a necessary and 
sufficient property of the goal state set Xg with respect to the map /. 

Theorem 12. Let f : X" — )■ X" be a mapping, let Cr denote the set of all cycles of the DES and let 
Xg C X" be a goal state set. Then, reachability of Xg with respect to f is given if and only if for all 
cycles c 6 Cr, there is a state ieX g that belongs to c. Denoting l as the longest open path and t as 
the length of the longest cycle of the DES, X g is reachable from any x £ X" in at most l + t — 1 steps. 

Algorithmically the verification of reachability for a given DES without controls with the 
mapping / and the goal state set X g can be done based on the knowledge of the number v of 
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cycles of the DES . First, it has to be noted that the requirement |X g | > v for the cardinality of 
Xg is a necessary requirement. If this condition is fulfilled, the following procedure performs 
the reachability verification. 

Algorithm 13. Input: Mapping f , goal state set X g , cycle count v 

1. Remove all states on trees from X g 

2. if v = 1 and X g ^ 

return rechability verification successful 

3. Pick x G X g 

Compute all states X g C X g on the same cycle as x 

X S = X g ~ X g 

4. v = v - 1 

5. if |X g | > v 

go to 2. 
else 

return reachability verification fails 

That is, Algorithm 13 checks if the states in X g cover each cycle of the DES. To this end, the 
algorithm successively picks states from Xg and removes all states in the same cycle from Xg. 
With the removal of each cycle, the variable v that represents the number of cycles of the DES 
that were not covered by states in Xg, yet, is decremented. Thereby, reachability is violated as 
soon as there are more remaining cycles V than remaining states in Xg. 

Next, stability for DES with no controls as in (5) is considered. In view of the previous 
discussion, stability requires that all states in all cycles of the DES belong to the goal set Xg. 
In that case, it holds that whatever start state x 6 X" is chosen, at most / steps are required 
to lead x to a cycle that belongs to X g . In contrast, it is clear that whenever there is a state 
x £ X" — Xg that belongs to a cycle of the DES, then the condition in (5) is violated for all 
states in the same cycle. Hence, the formal stability result for DES with no controls is as 
follows. 

Theorem 14. Let f : X" x X" be a mapping and let X g C X" be a goal state set. Then, stability of 
Xg with respect to f is given if and only ifXg contains all cyclic states of the DES with the mapping f. 
Denoting l as the longest open path of the DES, X g is reached from any x G X" in at most l steps. 

For the algorithmic verification of stability, a slight modification of Algorithm 13 can be used. 
It is only required to additionally check if the set X„ computed in step 3. contains all states of 
the respective cycle. In the positive case, the algorithm can be continued as specified, whereas 
the modified algorithm terminates with a violation of stability if X ? does not contain all states 
of a cycle. 

In summary, both reachability and stability of DES with no controls with respect to a given 
goal state set X g can be formulated and algorithmically verified in terms of the cycle structure 
of the DES. Moreover, it has to be noted that stability is more restrictive than reachability. 
While reachability requires that at least one state in each cycle of the DES belongs to Xg, 
stability necessitates that all cyclic states of the DES belong to X g . 



Note that v can be computed for LMS according to Subsection 2.1.2. 
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2.2.2 Reachability and stability under control 

The results in this subsection are valid for arbitrary finite fields. However, we will state the 
results with respect to the (for applications most relevant) Boolean finite field F2. Moreover, 
the focus of this subsection is the specialization of (4) to the case of controlled LMS with the 
following form 

x(k + l) = Ax(k) + Bu(k), k e N (17) 

with the matrices AeF 2 " x "andBeF™: 

k-l 
Vx G Fj(3Jfc G N and controls w(0), . . . ,u(k - 1) e W" s.t. A k x + £ A k ~ x ^Bu{j) G X g . (18) 

/=0 

In analogy to the classical reachability condition for linear discrete time systems that are 
formulated over the field R ((Sontag, 1998)), the following definition is sufficient for (18). 

Definition 15. The LMS in (17) is denoted as reachable if for any i,ie F!?, there exists a k G N and 
controls w(0), m(1), . . . ,u(k — 1) such that x{k) = x. If there is a smallest number I G N such that 
the above condition is fulfilled for any i,i£ FJJ and k = l, then the LMS is l-reachable. 

That is, if an LMS is reachable, then the condition in (18) is fulfilled for any given goal set 
Xg. To this end, the notion of controllability that is established for linear discrete time systems 
[Theorem 2 in (Sontag, 1998)] is formulated for LMS. 

Theorem 16. The LMS in (17) is controllable if and only if the pair (A, B) is controllable, that is, the 
matrix R with 

R= [B AB A 2 B ■ ■ ■ A"- 1 B] (19) 

has full rank n. Moreover, the LMS is I -controllable if and only if Rj = [B AB ■ ■ ■ A' _1 B] has full 
rank n for an I G {1, ..., n}. 

Noting the equivalence of controllability and reachability for linear discrete time systems 
as established in [Lemma 3.1.5 in (Sontag, 1998)], /-reachability for LMS can be verified by 
evaluating the rank of the matrix R[. In case an LMS is /-reachable, an important task is to 
determine an appropriate control input that leads a given start state x to the goal state set Xg. 
That is, for some x G Xg the controls u(0), . . .,u(l — 1) G W' have to be computed such that 
x = A l x + i: l j zlA l - 1 -iBu(i). 

To this end, a particular matrix L G F? xn is defined in analogy to [p. 81 in (Wolovich, 
1974)]. Denoting the column vectors of the input matrix B as b\, . . . , b m (which, without loss 
of generality, are linearly independent), that is, B = [bj • • • b m ] , L is constructed by choosing 
n linearly independent columns from R; with the following arrangement: 

L=[biAbi ■■■ A'"- 1 ^ b 2 Ab 2 ■■■ A^~ x b 2 ••■ b„, Ab m ■■■ Al l «<- l b m ] . (20) 

In this expression, the parameters \i\,...,y m that arise from the choice of the linearly 
independent columns of R\ are the controllability indices of the LMS (A, B). Without loss of 
generality it can be assumed that the controllability indices are ordered such that ]l\ < • • • < 
u m , in which case they are unique for each LMS. The representation in (20) allows to directly 
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compute an appropriate control sequence that leads x to a state x £ X g . It is desired that 
1-1 



A'x + £ A'- 1 -' [h 
;'=o 



bm] u(j) 



A l x- 



[&!••. A l -% ■ ■ ■ b m ■ ■ ■ A l - l b m ] [ut{l - 1) ■ ■ ■ «i(0) ■ ■ ■ u m (l -!)••• M m (0)] f 



In the above equation, u,(j) denotes the 7-th component of the input vector u(J) £ LP" at step 
j. Next, setting all M; (j) = for i = 1, . . . , m and j < I — 1 — jij, the above equation simplifies 
to 



A l x+[b x 



■Af< 



,-1 



b m ] ["l('-l)- ■ '"lC-fl)- ■ ■"«,(?-!)■ ■ •Mm('-^m)] f 



= A'x+L [u x {1 - 1) • ■ ■ «!(Z - \i\)--- u m (l - 1) • • ■ u„,(/ - i(„,)] f 
Since L is invertible, the remaining components of the control input evaluate as 

[ui{l - 1) ■ ■ ■ «j(Z - jli) ■ ■ ■ U m (l - 1) ■ ■ ■ U m (l - fl m )] t = L _1 (x - A'x). 

Remark 17. At this point, it has to be noted that the presented procedure determines one of the possible 
control input sequences that lead a given x to x £ X g . In general, there are multiple different control 
input sequences that solve this problem. 

Considering stability, it is required to find a control input sequence that finally leads a given 
start state to the goal state set Xg without leaving the goal state set again. For DES with no 
controls that are described in Subsection 2.2.1, stability can only be achieved if all cyclic states 
of an LMS are contained in Xg. In the case of LMS with control, this restrictive condition 
can be relaxed. It is only necessary that the goal set contains at least one full cycle of the 
corresponding system with no controls (for B = 0), that is, all states that form at least one 
cycle of an LMS. If /-reachability of the LMS is given, then it is always possible to reach this 
cycle after a bounded number of at most / steps. 

Corollary 18. The LMS in (17) is stable if it is I -reachable for an I G {1, ..., n} and X g contains all 
states of at least one cycle of the autonomous LMS with the system matrix A. 

Next, it is considered that /-reachability is violated for any / in Corollary 18. In that case, the 
linear systems theory suggests that the state space is separated into a controllable state space 
and an uncontrollable state space, whereby there is a particular transformation to the state 
y = T~ x that structurally separates both subspaces as follows from [p. 86 in ((Wolovich, 
1974))]. 



y{k) = TAf- l y{k - 1) + TBu(k - 1) 



Vcik) 

•/c-(fc) 



A c A cc 
Ac 



y(fc-i) 



u(fc-l). (21) 



This representation is denoted as the controller companion form with the controllable subsystem 
(A c , B c ), the uncontrollable autonomous subsystem with the matrix Ag and the coupling 
matrix A C c- 



Then, the following result is sufficient for the reachability of a goal state y 



Vc 



from a start 



state y 



, whereby y c ,yc and yc,ys denote the controllable and reachable part of the 



respective state vectors in the transformed coordinates. 
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Theorem 19. Assume that an uncontrollable LMS is given by its controller companion form in (21) 



be a start state and y 



G y g 



and assume that the pair (A c , B c ) is l-controllable. Let y = 
be a goal state. Then, y is reachable from y in k steps if 

• k>l 

• ye = A\y-c 

Theorem 19 constitutes the most general result in this subsection. In particular, Theorem 16 is 
recovered if the uncontrollable subsystem of the LMS does not exist and k > I. 
Finally, the combination of the results in Theorem 19 and Corollary 18 allows to address the 
issue of stability in the case of an uncontrollable LMS. 

Corollary 20. Consider an LMS in its Kalman decomposition (21). The LMS is stable if it holds for 
the uncontrollable subsystem that all states in cycles of A e are present in the uncontrollable part y e of 
the goal states y g Y g , whereby each cycle in the uncontrollable subsystem has to correspond to at least 
one cycle of the complete state y in Y g . 

2.3 Cycle sum assignment 

In regard of Section 2.1, imposing a desired cycle sum on an LMS requires to alter the system 
matrix in such a way that it obtains desired invariant polynomials that generate the desired 
cycle sum. Under certain conditions, this task can be achieved by means of linear state 
feedback of the form u(k) =Kx(k) with KeF™". 

Since the specification of a cycle sum via periodic polynomials will usually entail the need to 
introduce more than one non-unity invariant polynomial, invariant polynomial assignment 
generalizes the idea of pole placement that is wide-spread in the control community. The 
question to be answered in this context is: what are necessary and sufficient conditions for 
an LMS such that a set of invariant polynomials can be assigned by state feedback? The 
answer to this question is given by the celebrated control structure theorem of Rosenbrock in 
[Theorem 7.2.-4. in (Kailath, 1980)]. Note that, in this case, the closed-loop LMS assumes the 
formx(fc + l) = (A + BK)x(k). 

Theorem 21. Given is an n-dimensional and n-controllable LMS with m inputs. Assume that the 
LMS has the controllability indices }i\ > . . . > u m . Let cyj € F2[A] with ci + i K \c^ K , i = l,...,m — 
1, and J2T=1 d e g( c i',jc) = n be the desired non-unity monic invariant polynomials. Then there exists 
a matrix K g F™*" such that A + B K has the desired invariant polynomials cyj if and only if the 
inequalities 

k k 
£deg(q /K )>£>, k = l,2 m (22) 

(=1 ;=1 

are satisfied. 

Remark 22. The sum of the invariant polynomial degrees and the n-controllability condition guarantee 
that equality holds for k = m. However, the choice of formulation also includes the case of systems with 
single input m. In this case, Rosenbrock's theorem requires n-controllablity when a desired closed-loop 
characteristic polynomial is to be assigned by state feedback. Furthermore, the theorem indicates that at 
most m different invariant polynomials may be assigned in an LMS with m inputs. 

Assigning invariant polynomials is equivalent to assigning the non-unity polynomials of the 
Smith canonical form of the closed-loop characteristic matrix XI — (A + BK). It has to be 
noted that although meeting the assumptions of the control structure theorem with the desired 
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closed-loop Smith form, the extraction of the corresponding feedback matrix K is not a trivial 
task. The reason for this is that, in general, the structure of the Smith form of A/ — (A + BK) 
does not necessarily agree with the controllability indices of the LMS, which are preserved 
under linear state feedback. However, there is a useful reduction of the LMS representation 
based on the closed loop characteristic matrix in the controller companion form as shown in 
[Theorem 5.8 in (Reger, 2004)]. 

Theorem 23. Given is an n-dimensional n-controllable LMS in controller companion form (21) with 
m inputs and controllability indices \i\, . . . , \i m . Let D^ G F2[A] mx '" denote the polynomial matrix 



D*(A) := A(A) 



X,nonzero 



PW, 



where Ap g 

K,nonzero 

system matrix A c + 



f mxn 



contains the m non-zero rows of the controllable part of the closed-loop 
B c K in controller companion form, and P e F 2 [A]" xm , A G F 2 [A] mxm are 



( 



P(A) 



A'"- 1 
1 

A^-i 



V 





1 



\ 



A(A) 



/A''i 
' w 



^ 



(23) 



\ ■■■ A' ( »7 



Then the non-unity invariant polynomials of AI — (A c + B C K), XI — (^nonzero + K) and Dg-(A) 
coincide, whereby A nonZ ero contains the nonzero rows of the controllable part A c of the original system 
matrix. 

Theorem 23 points out a direct way of realizing the closed-loop LMS y(k + 1) = (A c + 
B c K)y(k) with desired invariant polynomials by means of specifying D^(A). That is, if 
an appropriate Dp can be found, then a linear state feedback matrix K in the transformed 
coordinates can be directly constructed. Simple manipulations first lead to 



K,nonzero 



P(A) = Djf(A) - A(A) 



(24) 



from which Ag- nonzero can be determined by comparison of coefficients. Then, by Theorem 
23, the feedback matrix 



K - A nonze ro ^K,nonzero ( 25 ) 

is obtained. Finally, the inverse coordinate transformation from the controller companion 
form to the original coordinates yields 

K = KT. (26) 

Hence, it remains to find an appropriate matrix D^. To this end, the following definitions are 
employed. 
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Definition 24. Let M G F2[A]" xm be arbitrary. The degree of the highest degree monomial in A 
within the i-th column o/M(A) is denoted as the i-th column degree of M and denoted by col,(M). 

Definition 25. Let M £ F2[A]" xm be arbitrary. The highest column degree coefficient matrix 
T(M) € F 2 " xm is the matrix whose elements result from the coefficients of the highest monomial 
degree in the respective elements o/M(A). 

Then, the following procedure leads to an appropriate D^. Starting with a desired cycle 
sum for the closed-loop LMS, an appropriate set of invariant polynomials - as discussed 
in Section 2.1 - has to be specified. Next, it has to be verified if the realizability condition 
of Rosenbrock's control structure theorem for the given choice of invariant polynomials is 
fulfilled. If the polynomials are realizable then D^(A) is chosen as the Smith canonical form 
that corresponds to the specified closed-loop invariant polynomials. In case the column 
degrees of D^(A) coincide with the respective controllability indices of the underlying LMS, 
that is, col, (D^) = itj for 1 = 1,..., m, it is possible to directly calculate the feedback matrix K 
according to (26). Otherwise, it is required to modify the column degrees of D^(A) by means 
of unimodular left and right transformations while leaving the invariant polynomials of D^ 
untouched. This procedure is summarized in the following algorithm. 

Algorithm 26. Input: Pair (A C ,B C ) in controller companion form® controllability indices \i\ > 
■■■ > f'm, polynomials c iK G F2[A], i = \,...,m with C; + i K \c; K , j = \,...,m— 1 and 
E™ 1 deg( C ,, K ) = «. 

1. Verify Rosenbrock's structure theorem 

if the inequalities in Theorem 21 are fulfilled 

go to step 2. 
else 

return "Rosenbrock's structure theorem is violated." 

2. DefineD*(X) := diag(ci /K , . . . , c m)K ) 

3. Verify if the column degrees o/D*(A) and the controllability indices coincide 
if col,(D*) = p^ i = 1, . . . ,m 

go to step 6. 
else 

Detect the first column of D*(A) which differs from the ordered list of controllability indices, 
starting with column 1. Denote this column col„(D*) (deg(col u (D*)) > p u ) 
Detect the first column o/D*(A) which differs from the controllability indices, starting with 
column m. Denote this column coLj(D*) (deg(coLj(D*)) < p.^) 

4. Adapt the column degrees ofD*(\) by unimodular transformations 

Multiply row^(D*) by A and add the result to row u (D*) — > new matrix D + (A) 
if deg(col„(D+)) = deg(col M (D*)) - 1 

D + ( A) — > new matrix D ++ (A) and go to step 5. 
else 

Define r:= deg(col u (D*)) - deg(col d (D*)) - 1 

Multiply coLj(D + ) by A'' and subtract result from col„(D + ) — > new matrix D ++ (A) . 



8 



If the LMS is not given in controller companion form, this form can be computed as in [p. 86 
(Wolovich, 1974)]. 
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5. SetD*{\) = (T(D++))- 1 D++ (A) and go to step 3. 

6. D#(A) := D*(A) and return D^(A) 

It is important to note that the above algorithm is guaranteed to terminate with a suitable 
matrix Dg- if Rosenbrock's structure theorem is fulfilled. For illustration, the feedback matrix 
computation is applied to the following example that also appears in (Reger, 2004; Reger & 
Schmidt, 2004). Given is an LMS over F 2 of dimension n = 5 with m = 2 inputs in controller 
companion form (that is, T = I), 



y(k + l) = A c y(k) + B c u(k), 



/0 1 0\ 

'00100* 

00000 

00001 

Vi 1 0/ 



(1 



0\ 
' 


00 

V01/ 



from which the matrix A r 



can be extracted. 



= (0 0^ 
~ [l 1 0y 
As a control objective, we want to assign the invariant polynomials Ci^(a) 



+ a 



l)(fl + l) 2 andc 2;K (a) 



1, that is, according to the example in Subsection 2.1.2 this goal 



is equivalent to specifing that the closed-loop LMS shall have 4 cycles of length 1, 2 cycles of 
length 2, 4 cycles of length 3 and 2 cycles of length 6. An appropriate state feedback matrix K 
is now determined by using (26) and Algorithm 26. 

-^ E deg(c,- K (A)) = 4 > E c t = 3 and E deg(c i/X (A)) = 5 > £ c; = 5 y/ 

i=\ ! = 1 ( = 1 ( = 1 



3., 4. 



D*(A) 

D+(A) 



(A 2 + A + 1)(A + 1) 2 
A + l 



A 4 + A 3 + A + l A 2 + A 
A + l 



A 4 + A 3 + A + 1 
A + l 



D++(A) 



A + l A 2 + A 
A 3 + A 2 A + l 



r^ 



1 

1 



D*(A) 



(r(D ++ ))- lD++ (A)^(^f ^a) 



3., 4., 6. 



D*(A) 



A 3 + A 2 A + l 
A + l A 2 + A 



With D^(A) the feedback matrix K can be computed. First, employing equation (24) yields 



K,nonzero 



/l OX 
' A < 

A 2 
1 

Vo a/ 



A 3 

A 2 



A(A) 



A 3 + A 2 A + l 
^ A + l A 2 + A 

D*(A) 



A 2 A + l 
A + l A 



5 Constructing the appropriate invariant polynomials based on the cycle structure desired is not always 
solvable and, if solvable, not necessarily a straightforward task (Reger & Schmidt (2004)). 
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and by comparison of coefficients results in A^ nonzero = ( ■, i n n i ) • This implies that 

K -Kt-(A« +A W-f° 0111 

A - KL - (A Knonzew + /inonzeroj J _ I 1 1 1 

3. Properties of Boolean monomial systems 10 

3.1 Dynamic properties, cycle structure and the loop number 

The aim of this section is that the reader becomes acquainted with the main theorems 
that characterize the dynamical properties of Boolean monomial dynamical systems without 
deepening into the technicalities of their proofs. We briefly introduce terminology and 
notation and present the main results. Proofs can be found in Delgado-Eckert (2008), and 
partially in Delgado-Eckert (2009a) or Colon-Reyes et al. (2004). 

Let G = {Vq, Eg, tiq) be a directed graph (also known as digraph). Two vertices a, b G Vq are 
called connected if there is a t 6 No and (not necessarily different) vertices V\, ..., Vt £ Vq such 
that 

a — > Vi — y V2 — >• ... — > v t — > b, 

where the arrows represent directed edges in the graph. In this situation we -write a ~-» s b, 
where s is the number of directed edges involved in the sequence from a to b (in this case 
s = t + 1). Two sequences a ~^ s b of the same length are considered different if the directed 
edges involved are different or the order at which they appear is different, even if the visited 
vertices are the same. As a convention, a single vertex a € Vq is always connected to itself 
a ~+q a by an empty sequence of length 0. A sequence a ~~* s b is called a path, if no vertex V{ is 
visited more than once. If a = b, but no other vertex is visited more than once, a ~+ s b is called 
a closed path. 

Let q £ N be a natural number. We denote with F,, a finite field with q elements, i.e. \W„ I = q. 
As stated in the introduction, every function h : FL' — > ¥q can be written as a polynomial 
function in n variables where the degree of each variable is less or equal to cf — 1. Therefore we 
introduce the exponents set (also referred to as exponents semiring, see below) E ? := {0, 1, ..., (q — 
2), (q — 1) } and define monomial dynamical systems over a finite field as: 

Definition 27. Let F,j be a finite field and n e N a natural number. A map f : F" — > F" is called an 
n-dimensional monk monomial dynamical system over F^ if for every i g {1, ..., n) there is a tuple 
(Fi\,..., Fi n ) £ F" such that 



f i (x)=x 1 Vl \..x%»Vx€T n q 



ii 



We will call a monk monomial dynamical system just monomial dynamical system. The matrix 
Fij g M(n x n;Fq) is called the corresponding matrix of the system f. 

Remark 28. As opposed to Colon-Reyes et al. (2004), we exclude in the definition of monomial 
dynamical system the possibility that one of the functions f is equal to the zero function. However, in 
contrast to Colon-Reyes et al. (2006), we do allow the case f = 1 in our definition. This is not a loss of 
generality because of the following: If we were studying a dynamical system f : F" — > F" where one 
of the functions, say fi, was equal to zero, then, for every initial state x g F" after one iteration the 
system would be in a state f(x) whose jth entry is zero. In all subsequent iterations the value of the jth 



10 Some of the material presented in this section has been previously published in Delgado-Eckert (2009b). 

11 M(n x n;Eq) is the set ofnxn matrices with entries in the set E,. 
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projection w> : F" — > F" 



entry would remain zero. As a consequence, the long term dynamics of the system are reflected in the 

n-\ 

rc/(y) : = (yi/-/yy-i/y/+i,.../y»)' 

And if is sufficient to study the system 



f : F"- 1 -> F- 1 



y ^ 



/ /i(yi/-vy;-i,0,y i+1 ,...,y„) ^ 

/jf-i(yi,...,y/-i/0,y ; - +1/ ...,y») 
//+i(yi/-/y/-i,0,y ; - +1/ ...,y„) 

V fn(yi,-,yj-i,o,yj+i,-,y n ) ) 



hi general, this system f could contain component functions equal to the zero function, since every 
component f that depends on the variable X; would become zero. As a consequence, the procedure 
described above needs to be applied several times until the lower dimensional system obtained does not 
contain component functions equal to zero. It is also possible that this repeated procedure yields the 
one dimensional zero function. In this case, we can conclude that the original system f is a fixed point 
system with (0, ...,0) € F" as its unique fixed point. The details about this procedure are described 
as the "preprocessing algorithm" in Appendix B of Delgado-Eckert (2008). This also explains why we 
exclude in the definition of monomial feedback controller (see Definition 62 in Section 3.2 below) the 
possibility that one of the functions /; is equal to the zero function. 

When calculating the composition of two monomial dynamical systems f,g : F„ — > W2 (i.e. 
the system fog : F" — > F" x i-> f(g(x))), one needs to add and multiply exponents. 
Similarly, when calculating the product f * g, where * is the component-wise multiplication 
defined as 

(f*g)i(x):=f t {x)gi(x) 

one needs to add exponents. However, after such operations, one may face the situation 
where some of the exponents exceed the value q — 1 and need to be reduced according to the 
well known rule «' = iiV(i6F^ This process can be accomplished systematically if we look 
at the power x- (where p > q) as a polynomial in the ring ¥„ [t] and define the magnitude 
redq(p) as the degree of the (unique) remainder of the polynomial division t ? ' -j- (t^ — t) 

in the polynomial ring F<Jt]. Then we can write x = x- ' V X\ £ Fn, which is a direct 
consequence of certain properties of the operator red„ (see Lemma 39 in Delgado-Eckert 
(2008)). In conclusion, the "exponents arithmetic" needed when calculating the composition of 
dynamical systems /, g : FL' — > K can be formalized based on the reduction operator red„(p). 
Indeed, the set Eo = {0, 1, ..., (q — 1)} C Z together with the operations of addition a © b : = 
red„(a + b) and multiplication a»b := red„(ab) is a commutative semiring with identity 1. We 
call this commutative semiring the exponents semiring of the field Fn. Due to this property, the 
set of all n-dimensional monomial dynamical systems over ¥„, denoted with MF,"(Fo), is a 
monoid (MFjj (Fq),o), where o is the composition of such systems. Furthermore, this set is 
also a monoid (MF"(F^),*) where * is the component-wise multiplication defined above. In 
addition, as shown in Delgado-Eckert (2008), these two binary operations satisfy distributivity 



properties, i.e. (MF" (Wq 



is a semiring with identity element with respect to each binary 
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operation. Moreover, Delgado-Eckert (2008) proved that this semiring is isomorphic to the 
semiring M(n x n; E„) of matrices with entries in E„. This result establishes on the one hand, 
that the composition / o g of two monomial dynamical systems /, g is completely captured 
by the product F • G of their corresponding matrices. On the other hand, it also shows that 
the component-wise multiplication / * g is completely captured by the sum F + G of the 
corresponding matrices. Clearly, these matrix operations are defined entry-wise in terms 
of the operations © and •. The aforementioned isomorphism makes it possible for us to 
operate with the corresponding matrices instead of the functions, which has computational 
advantages. Roughly speaking, this result can be summarized as follows: There is a bijective 
mapping 

Y : (M(n x n; £,),+,♦) -► (MF£(F,),*,o) 

which defines a one-to-one correspondence between matrices and monomial dynamical 
systems. The corresponding matrix defined above can therefore be calculated as Y _1 (/). This 
result is proved in Corollary 58 of Delgado-Eckert (2008), which states: 

Theorem 29. The semirings (M(n x n; En), +, ■) and (MF^(F^), *, o) are isomorphic. 

Another important aspect is summarized in the following remark 

Remark 30. Let Wq be a finite field and n,m,r 6 N natural numbers. Furthermore, let f e MF™(F ? ) 
andge MF^(F ? ) with 

f { (x) = xf 1 ...x F ri" V x e F", i = l,...,m 
g j (x) = x? 1 ...x%" , Vx€V™,j = l r 

where F 6 M(m x n;Eq) and G 6 M(r x m; Eq) are the corresponding matrices of f and g, 
respectively. Then for their composition g o f : F„ — ^ F„ it holds 

(g°f)k(x) = flxj^hVxe ¥ n ql k G {1 r} 

Proof. See Remark and Lemma 51 of Delgado-Eckert (2008). ■ 

The dependency graph of a monomial dynamical system (to be defined below) is an important 
mathematical object that can reveal dynamic properties of the system. Therefore, we turn our 
attention to some graph theoretic considerations: 

Definition 31. Let Mbea nonempty finite set. Furthermore, let n := \M\ be the cardinality of M. An 
enumeration of the elements of M is a bijective mapping a : M — > {1, ...,«}. Given an enumeration a 
of the set M we write M = {a\, —,a n }, where the unique element x e M with the property a(x) = i G 
{1, ...,n} is denoted as a,-. 

Definition 32. Let f 6 MF]f (F ? ) be a monomial dynamical system and G = (Vq, Eq, tzq) a digraph 
with vertex set Vq of cardinality \Vq\ = n. Furthermore, let F := Y _1 (/) be the corresponding matrix 
of f. The digraph G is called dependency graph of f iff an enumeration a : M — > {1, ..., n} of the 
elements of Vq exists such that V i,j G {l,...,n} there are exactly F;y directed edges a, — )■ a;intheset 

E G ,i.e. J7T G 1 ((fl„fly))| = F tj . 

It is easy to show that if G and H are dependency graphs of / then G and H are isomorphic. 
In this sense we speak of the dependency graph of/ and denote it by Gt = (Vc, Ec, rcc). 
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Definition 33. Let G = (Vq, Eq, Kq) be a digraph. Two vertices a,b £ Vq are called strongly 
connected if there are natural numbers s, t e N such that a -~^ s b and b ~~»j a. In this situation we 
write a ^ b. 

Theorem 34. Let G = (Vq, Eq, tzq) be a digraph. ^ is an equivalence relation on Vq called strong 
equivalence. The equivalence class of any vertex a g Vq is called a strongly connected component 
and denoted by a C Vq. 

Proof. This a well known result. A proof can be found, for instance, in Delgado-Eckert (2008), 
Theorem 68. ■ 

Definition 35. Let G = (Vq, Eq, tzq) be a digraph and a £ Vq one of its vertices. The strongly 
connected component a C Vq is called trivial iff a = {a} and there is no edge a —¥ a in Eq. 

Definition 36. Let G = (Vq, Eq, tzq) be a digraph with vertex set Vq of cardinality \Vq\ = n 
and Vq = {a.\, ...,a n } an enumeration of the elements ofVQ. The matrix A G M(n x n; No) ivhose 
entries are defined as 

Aj; := number of edges aj — ► a; contained in Eq 

for i,j = \,...,n is called adjacency matrix of G with the enumeration a. 

Remark 37. Let f e MFjj(Fq) be a monomial dynamical system. Furthermore, let Gt = (Vt, Et, 
Tif) be the dependency graph of f and Vt = {a-y, ..., a n } the associated enumeration of the elements of 
Vt. Then, according to the definition of dependency graph, F := Y _1 (/) (the corresponding matrix of 
f) is precisely the adjacency matrix of Gr with the enumeration a. 

The following parameter for digraphs was introduced into the study of Boolean monomial 
dynamical systems by Colon-Reyes et al. (2004): 

Definition 38. Let G = (Vq, Eq, tiq) be a digraph and a e Vq one of its vertices. The number 

£c( a ) '■= mm \ u — v \ 

is called the loop number of a. If there is no sequence of positive length from a to a, then Cq(o) is set to 
zero. 

Note that the loop number Cqi(o) of the vertex a in a graph G' = (Vq, Eq, tz'q) may have a 
different value. 

Lemma 39. Let G = (Vq, Eq, tzq) be a digraph and a € Vq one of its vertices. If a is nontrivial 
then for every be a it holds Cc(b) = £,Q(a). Therefore, we introduce the loop number of strongly 
connected components as 

Cg(V) := C G (a) 

Proof. See Lemma 4.2 in Colon-Reyes et al. (2004). ■ 

The loop number of a strongly connected graph is also known as the index of imprimitivity 
(see, for instance, Ptak & Sedlacek (1958)) or period (Denardo (1977)) and has been used in 
the study of nonnegative matrices (see, for instance, Brualdi & Ryser (1991) and Lancaster 
& Tismenetsky (1985)). This number quantizes the length of any closed sequence in a strongly 
connected graph, as shown in the following theorem. It is also the biggest possible "quantum", 
as proved in the subsequent corollary. 
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Theorem 40. Let G = (Vq, Eq, n G ) be a strongly connected digraph. Furthermore, let t := 
£g{Vg) > be its loop number and a g Vq an arbitrary vertex. Then for any closed sequence 
a ^ m a there is an a. G No such that m = at. 

Proof. This result was proved in Corollary 4.4 of Colon-Reyes et al. (2004). A similar proof 
can be found in Delgado-Eckert (2009b), Theorem 2.19. ■ 

Corollary 41. Let G = (Vq, Eq, tzq) be a strongly connected digraph such that Vq is nontrivial and 
Vq = {flj, ..., a n ) an enumeration of the vertices. Furthermore, let l\, ..., /j- G {1, ..., n} be the different 
lengths of non-empty closed paths actually contained in the graph G. That is, for every j e {1, ...,k} 
there is an a, e Vq such that a closed path a.{. ~~>/. a,-, exists in G, and the list l\,...,\ captures all 
different lengths of all occurring closed paths. Then the loop number Cq(Vq) satisfies 

£ G (V G )= S cd(h,...,l k ) 

Proof. This result was proved in Theorem 4.13 of Colon-Reyes et al. (2004). A slightly simpler 

proof can be found in Delgado-Eckert (2009b), Corollary 2.20. ■ 

The next results show how the connectivity properties of the dependency graph and, in 

particular, the loop number are related to the dynamical properties of a monomial dynamical 

system. 

Theorem 42. Let Fq be a finite field and f G MFJ'(F ? ) a monomial dynamical system. Then f is a 
fixed point system with (1, ..., l) f G F™ as its only fixed point if and only if its dependency graph only 
contains trivial strongly connected components. 

Proof. See Theorem 3 in Delgado-Eckert (2009a). ■ 

Definition 43. A monomial dynamical system f G MF^(F ? ) whose dependency graph contains 
nontrivial strongly connected components is called coupled monomial dynamical system. 

Definition 44. Let m G N be a natural number. We denote with D(m) := {d G N : d divides m) 
the set of all positive divisors ofm. 

Theorem 45. Let F2 be the finite field with two elements, f G MFj} (F2 ) a Boolean coupled monomial 
dynamical system and Gr = (Vt, Et, nA its dependency graph. Furthermore, let Gt be strongly 
connected with loop number t := Cq (VA > 1. Then the period number T (cf Section 1.1) of f 
satisfies 

T = C Gf (V f ) 

Moreover, the phase space off contains cycles of all lengths s G D(T). 

Proof. This result was proved by Colon-Reyes et al. (2004), see Corollary 4.12. An alternative 
proof is presented in Delgado-Eckert (2008), Theorem 131. ■ 

Theorem 46. Let F2 be the finite field with two elements, f G MF" (F2 ) a Boolean coupled monomial 
dynamical system and Gr = (Vt, Et, nA its dependency graph. Furthermore, let Gt be strongly 
connected with loop number t := Cq (VA > 1. In addition, let s G N be a natural number and 
denote by Z s the number of cycles of length s displayed by the phase space of f. Then it holds for any 

' d -Lieu WV Z i ifd g D(f) 

Qifdt D(t) 
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Proof. See Theorem 132 in Delgado-Eckert (2008). ■ 

Theorem 47. Let F2 be the finite field with two elements, f G MF" (F2 ) a Boolean coupled monomial 
dynamical system and Gt = (Vr, Et, nA its dependency graph, f is a fixed point system if and only 
if the loop number of each nontrivial strongly connected component of Gr is equal to 1. 

Proof. This result was proved in Colon-Reyes et al. (2004), see Theorem 6.1. An alternative 
proof is presented in Delgado-Eckert (2009a), Theorem 6. ■ 

Remark 48. As opposed to the previous two theorems, the latter theorem does not require that Gr 
is strongly connected. This feature allows us to solve the stabilization problem (see Section 3.3) for a 
broader class of monomial control systems (see Definition 54 in Section 3.2). 

Lemma 49. Let G = (Vq, E g , tz g ) be a strongly connected digraph such that V G is nontrivial. 
Furthermore, let t := Cq(Vq) > be its loop number. For any a,b G Vq the relation « defined by 

a « b :-4=> 3 a sequence a ~» a f b with a G No 

is an equivalence relation called loop equivalence. The loop equivalence class of an arbitrary vertex 
a G Vq is denoted by a. Moreover, the partition of Vq defined by the loop equivalence fa contains 
exactly t loop equivalence classes. 

Proof. See the proofs of Lemma 4.6 and Lemma 4.7 in Colon-Reyes et al. (2004). ■ 

Definition 50. Let G = (Vq, Eq, tzq) be a digraph, a £ Vq an arbitrary vertex and m G IN a 
natural number. Then the set 

N m (a) := {be V G :3a~^ m b] 

is called the set of neighbors of order m. 
Remark 51. From the definitions it is clear that 

a= 1J N at (a) 

Theorem 52. Let G = (Vq, Eq, tzq) be a strongly connected digraph such that Vq is nontrivial. 
Furthermore, let t := Cq(Vq) > be its loop number and a C Vq an arbitrary loop equivalence class 
ofV G . Then for any b,b' £ a the following holds 

1. N m (b) n N m t (b 1 ) = <Z>for m, m'eN such that 1 < m, m' < t and m £ ml '. 

2. N m (b) n a = Q)for m e N such that 1 < m < t. 

3. For every fixed m e N such that 1 < m < t3 c G Vq : U N m (b) = c. 

Proof. See Theorem 111 in Delgado-Eckert (2008). ■ 

Remark 53. It is worth mentioning that since Vq is strongly connected and nontrivial, N„,(b) ^ 
V m € N, b G Vq. Moreover, from (1) in the previous theorem it follows easily 

\J N m (b) J n |J N ml (b) J = <Z)for m,m' G N such that 1 < m,m' < t and m ^ m' 
bed ) \bed ) 
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Fig. 1. Strongly connected dependency graph Gr = (Vf, Ec, ni) with loop number 
Cq AVr) = 6 of a 24-dimensional Boolean monomial dynamical system / G MF^f (F2). 
Circles (blue) demarcate each of the six loop equivalence classes. Essentially, the dependency 
graph is a closed path of length 6. 

and because of (2) in the previous theorem clearly 

a= \jN t (b) 

tea 

Given one loop equivalence class a C Vq, the set of all the t loop equivalence classes can be ordered in 
the following manner 

% :=a, a t+1 = \J N l (b),...a i+j = \J Nj(b),...a i+t ^ = \J N t ^(b) 

bta, bGflf b€at 

For any c G \J N t _i (b) it must hold Nj (c) C a i (ifN\ (c) n a, ■ ^ with j 7^ i, then 3; = a;). Thus, 

tea; 

the graph G can be visualized as (see Fig. 1) 

«i =4 Oi+l =4 • • • =i a i+ j =4 «(i+;+l) mod f =J »• =J «f+f-l =4 «(,'+() mo d f 

Due to the fact a = \J Nt (b) V a G Vg, roe cam conclude that the claims of the previous lemma still 

bea 

hold if the sequence lengths m and ml are replaced by the more general lengths kt + m and X't + ml ' , 
where A, A' G N. 

3.2 Boolean monomial control systems: Control theoretic questions studied 

We start this section with the formal definition of a time invariant monomial control system 
over a finite field. Using the results stated in the previous section, we provide a very compact 
nomenclature for such systems. After further elucidations, and, in particular, after providing 
the formal definition of a monomial feedback controller, we clearly state the main control 
theoretic problem to be studied in Section 3.3 of this chapter. 

Definition 54. Let F ? be a finite field, n G N a natural number and m G INo a nonnegative integer. 
A mapping g : F" x F™ — > F" is called time invariant monomial control system over F fl if for 
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every i g {1, ..., n) there are two tuples (An, ..., A,„) G E" and (B;i, ..., B,„,) G E" 1 such that 
gi (x,u) = x A '\..x A "'uf...u^' V (x,u) G F£ x F* 

Remark 55. Ik fne case m = 0, 7t>e have F'" = F9 = {()} (the set containing the empty tuple) and 
thus F" x F™ = F" x F^ = F" x {()} = F" In other words, g is a monomial dynamical system 
over ~¥q. From now on we will refer to a time invariant monomial control system over F, f as monomial 
control system over F<j. 

Definition 56. Let Xbe a nonempty finite set and n,! 6N natural numbers. The set of all functions 
f : X 1 ->■ X" is denoted with Ff(X). 

Definition 57. Let F ? be a finite field and l,m,n G N natural numbers. Furthermore, let Eq be the 
exponents semiring of¥q and M(n x /; Eq) the set of n x / matrices with entries in Eq. Consider the 
map 

T : F' m (fq)xM(nxl;Eq)^F'„',(TFq) 
(f,A) ^ Y A (f) 

where T A (f) is defined for every x G F'" and i G {1, ...,«} by 

T A (f)(x) i :=f 1 (x) A -...f l (x)^ 
We denote the mapping T A (f) G F^'(Fq) simply Af. 

Remark 58. Let I = m, id G F™(Fq) be the identity map (i.e. idj(x) = x, V i G {l,...,m}) and 
A G M(n x m;Eq) Then the following relationship between the mapping Aid G F,"(Fq) and any 
feF%(Fq) holds 

Aid(f(x)) = Af(x)VxeP» 

Remark 59. Consider the case I = m = n. For every monomial dynamical system f G MF,"(Fg) C 
FH(¥q) with corresponding matrix F := Y _1 (/) G M(n x n;Eq) it holds Fid = f. On the other 
hand, given a matrix F G M(n x n;Eq) we have Y _1 (F;d) = F. Moreover, the map Y : F^(¥q) x 
M(n xn;£ ( )4 F,"(F ? ) is an action of the multiplicative monoid M(n x n; Eq) on the set F"(F ? ). It 
holds namely, that 12 If = / V/ e F£(F ? ) (Wiic/z is irarcaZJ and (A ■ B)f = A(Bf) V / G F£(F ? ), 
A, B G M(n x n;Eq).To see this, consider 

((A ■ B)f)i(x) = f l (x^ A - B ^...f n (x)( A ^- = flf j ( x )(Aa*By®...®A in .B nl ) 

= (AidoBid)i(f(x)) = (Aid)i(Bid(f(x))) 
= (Aid) i (fB(x)) = (A(Bf)),(x) 

whereid G F^(F^) is the identity map (i.e. idi(x) = x,- V 2 G {l,...,n}J. (c/. with the proof of Theorem 
29). As a consequence, MF%(Fq) is the orbit in F£(Fq) of id under the monoid M(n x n;Eq). In 
particular (see Theorem 29), we have 

(F ■ G)id = F(Gid) = fog 



I G M(n x ft; Eq) denotes the identity matrix. 
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where g g MF"(F ? ) is another monomial dynamical system with corresponding matrix G := 

f- 1 (g)eM(nxn;E q ). 

Lemma 60. Let F ? be a finite field, n g N a natural number and m G No a nonnegative integer. 
Furthermore, let id e F}" ™ (F ? ) fee t/ie identity map (i.e. idj(x) = Xj V i G {l,...,n + in}) and 
g : F" x F"' — ► F" a monomial control system over ¥„. Then there are matrices A g M(n x n; £ ? ) 
rtnd B e M(« x m; Eq) such that 

({A\B)id)(x,u) = g(x,u) V (x,u) e F» x F™ 

w/zere (^4|B) £ M(n x (n + m); £ ? ) is the matrix that results by writing A and B side by side. In this 
sense we denote g as the monomial control system (A, B) with n state variables and m control inputs. 

Proof. This follows immediately from the previous definitions. ■ 

Remark 61. If the matrix B £ M(n x m; Eq) is equal to the zero matrix, then g is called a control 
system with no controls. In contrast to linear control systems (see the previous sections and also 
Sontag (1998)), when the input vector u G F'" satisfies 

u = i:=(i,...,i) t e Wq" 

then no control input is being applied on the system, i.e. the monomial dynamical system over F^ 

<r ■■ V'^V'q' 
x^ g(x,l) 

satisfies 

o-(x) = ((A\0)id)(x,u) V (x,u) eFjx F™ 

where G M(n x m; Eq) stands for the zero matrix. 

Definition 62. Let Fg be a finite field and n, m S N natural numbers. A monomial feedback 
controller is a mapping 

f : F» -> F™ 

such that for every i 6 {!,... ,m) there is a tuple (F (1 , ..., F,„ ) £ £" such that 

f i (x)=xfK..x%»Vx€F n q 

Remark 63. We exclude in the definition of monomial feedback controller the possibility that one of the 
functions // is equal to the zero function. The reason for this will become apparent in the next remark 
(see below). 

Now we are able to formulate the first control theoretic problem to be addressed in this section: 

Problem 64. Let Wq be a finite field and n, m £ N natural numbers. Given a monomial control system 
g : F" x F 1 " — ¥ F" with completely observable state, design a monomial state feedback controller 
f : Fj! — > F'" such that the closed-loop system 

h : F«->F» 
x i-> g{x,f{x)) 
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has a desired period number and cycle structure of its phase space. What properties has g to fulfill for 
this task to be accomplished? 



Remark 65. 


Note that every component 










hi : 


F? 


-*■ F,,, i = 


1,.. 




x n- 


gi( 


*//(*)) 




is a nonzero monic monomial function, i.e. 


the 


mapping h 


■ F" 



F" is a monomial dynamical system 
over Fa. Remember that we excluded in the definition of monomial feedback controller the possibility 
that one of the functions /,- is equal to the zero function. Indeed, the only effect of a component /, = 
in the closed-loop system h would be to possibly generate a component hi = 0. As explained in Remark 
28 of Section 3.1, this component would not play a crucial role determining the long term dynamics of 
h. 

Due to the monomial structure ofh, the results presented in Section 3.1 of this chapter can be used to 
analyze the dynamical properties ofh. Moreover, the following identity holds 

h = (A + B- F)id 

where F G M(m x n; En) is the corresponding matrix of f (see Remark 30), (A, B) are the matrices in 
Lemma 60 and id £ F* (F^). To see this, consider the mapping 

u : F™^F» 

u h+ g(i,u) 

where 1 e F" From the definition of g it follows that \i e MF,",(F ? ). Now, since f £ MF"'(Wq), by 
Remark 30 we have for the composition y. o f : F" — )■ F" 

uof= (B-F)id 

Now its easy to see 

h= (A + B-F)id 

The most significant results proved in Colon-Reyes et al. (2004), Delgado-Eckert (2008) 
concern Boolean monomial dynamical systems with a strongly connected dependency graph. 
Therefore, in the next section we will focus on the solution of Problem 64 for Boolean 
monomial control systems g : FS X F™ —¥ F^ with the property that the mapping 

a : Ff ->■ F| 

x^ S (x,i) 

has a strongly connected dependency graph. Such systems are called strongly dependent 
monomial control systems. If we drop this requirement, we would not be able to use Theorems 
45 and 46 to analyze h regarding its cycle structure. However, if we are only interested in 
forcing the period number of h to be equal to 1, we can still use Theorem 47 (see Remark 48). 
This feature will be exploited in Section 3.3, when we study the stabilization problem. 
Although the above representation 

h = (A + B-F)id 
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of the closed loop system displays a striking structural similarity with linear control 
systems and linear feedback laws, our approach will completely differ from the well known 
"Pole-Assignment" method. 

3.3 State feedback controller design for Boolean monomial control systems 

Our goal in this section is to illustrate how the loop number, a parameter that, as we 
saw, characterizes the dynamic properties of Boolean monomial dynamical systems, can be 
exploited for the synthesis of suitable feedback controllers. To this end, we will demonstrate 
the basic ideas using a very simple subclass of systems that allow for a graphical elucidation 
of the rationale behind our approach. The structural similarity demonstrated in Remark 53 
then enables the extension of the results to more general cases. A rigorous implementation of 
the ideas developed here can be found in Delgado-Eckert (2009b). 

As explained in Remark 53, a Boolean monomial dynamical system with a strongly connected 
non-trivial dependency graph can be visualized as a simple cycle of loop-equivalence classes 
(see Fig. 1). In the simplest case, each loop-equivalence class only contains one node and 
the dependency graph is a closed path. A first step towards solving Problem 64 for strongly 
dependent Boolean monomial control systems g : Fj x F™ — ¥ Fj would be to consider the 
simpler subclass of problems in which the mapping 

<r : Ff ->■ Ff 
x^ g(x,l) 

simply has a closed path of length n as its dependency graph (see Fig. 2 a for an example 
in the case n = 6). By the definition of dependency graph and after choosing any monomial 
feedback controller f : FS — ► F™, it becomes apparent that the dependency graph of the 
closed-loop system 

h f : Fg ->■ F£ 
x i-> g(x,f(x)) 

arises from adding new edges to the dependency graph of a. Since we assumed that the 
dependency graph of a is just a closed path, adding new edges to it can only generate new 
closed paths of length in the range 1, . . . , n — 1. By Corollary 41, we immediately see that the 
loop number of the modified dependency graph (i.e., the dependency graph of hi) must be a 
divisor of the original loop number. This result is telling us that no matter how complicated 
we choose a monomial feedback controller / : F, — ► F™, the closed loop system hi will 
have a dependency graph with a loop number £.' which divides the loop number C of the 
dependency graph of a. This is all we can achieve in terms of loop number assignment. When a 
system allows for assignment to all values out of the set D(C), we call it completely loop number 
controllable. We just proved this limitation for systems in which o~ has a simple closed path 
as its dependency graph. However, due to the structural similarity between such systems 
and strongly dependent systems (see Remark 53), this result remains valid in the general case 
where a has a strongly connected dependency graph. 

Let us simplify the scenario a bit more and assume that the system g has only one control 
variable u (i.e., g : F 2 ! X F2 — > FJ?) and that this variable appears in only one component 
function, say g^. As before, assume u has a simple closed path as its dependency graph. Under 
these circumstances, we choose the following monomial feedback controllers: /, : F? — > F2, 
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/<(*) 



1, ..., n. When we look at the closed-loop systems 



h 



■Si 



¥l 



x H- g{x,fj{x)) 



and their dependency graphs, we realize that the dependency graph of he. corresponds to the 
one of C with one single additional edge. Depending on the value of i under consideration, 
this additional edge adds a closed path of length I in the range / = 1, .., n — 1 to the dependency 
graph of o~. In Figures 2 b-e, we see all the possibilities in the case of n = C = 6, except for 
/ = 1 (self-loop around the kth node). 




C = 6 









C = 3 





Fig. 2. Loop number assignment through the choice of different feeback controllers. 

We realize that with only one control variable appearing in only one of the components of 
the system g, we can set the loop number of the closed-loop system hf. to be equal to any 
of the possible values (out of the set D(C)) by choosing among the feedback controllers //, 
i = 1, ..., n, defined above. This proves that the type of systems we are considering here are 
indeed completely loop number controllable. Moreover, as illustrated in Figure 2 f, if the 
control variable U would appear in another component function of g, we may loose the loop 
number controllability. Again, due to the structural similarity (see Remark 53), this complete 
loop number controllability statement is valid for strongly dependent systems. 
In the light of Theorem 47 (see Remark 48), for the stabilization problem we can consider 
arbitrary Boolean monomial control systems g : FJJ x F™ — > F!/, maybe only requiring the 
obvious condition that the mapping o~ is not already a fixed point system. Moreover, the 
statement of Theorem 47 is telling us that such a system will be stabilizable if and only if the 
component functions gi depend in such a way on control variables u,, that every strongly 
connected component of the dependency graph of a can be forced into loop number one by 
incorporating suitable additional edges. This corresponds to the choice of a suitable feedback 
controller. The details and proof of this stabilizability statement as well as a brief description 
of a stabilization procedure can be found in Delgado-Eckert (2009b). 



13 Note that in contrast to the definition of stability introduced in Subsection 1.2.1, in this context we refer 
to stabilizability as the property of a control system to become a fixed point system through the choice 
of a suitable feedback controller. 
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4. Conclusions 

In this chapter we considered discrete event systems within the paradigm of algebraic state 
space models. As we pointed out, traditional approaches to system analysis and controller 
synthesis that were developed for continuous and discrete time dynamical systems may not 
be suitable for the same or similar tasks in the case of discrete event systems. Thus, one of 
the main challenges in the field of discrete event systems is the development of appropriate 
mathematical techniques. Finding new mathematical indicators that characterize the dynamic 
properties of a discrete event system represents a promising approach to the development of 
new analysis and controller synthesis methods. 

We have demonstrated how mathematical objects or magnitudes such as invariant 
polynomials, elementary divisor polynomials, and the loop number can play the role of the 
aforementioned indicators, characterizing the dynamic properties of certain classes of discrete 
event systems. Moreover, we have shown how these objects or magnitudes can be used to 
effectively address controller synthesis problems for linear modular systems over the finite 
field F2 and for Boolean monomial systems. 

We anticipate that the future development of the discrete event systems field will not only 
comprise the derivation of new mathematical methods, but also will be concerned with the 
development of efficient algorithms and their implementation. 
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1. Introduction 

The dynamical systems with discrete time and delay are obtained by the discretization of the 
systems of differential equations with delay or by modeling some processes in which the time 
variable is n G IV and the state variables at the moment n — m, where m G N, m > 1, are taken 
into consideration. 
The processes from this chapter have as mathematical model a system of equations given by: 

X n +\ = f \X n , x n — m , a), (1) 

where x„ = x{n) G RF, x n - m = x(n - m) <E R? , a. £ R and / : W X RP x R -¥ R? is a 
seamless function, n, m G N with m > 1. The properties of function f ensure that there is 
solution for system (1). The system of equations (1) is called system with discrete-time and delay. 
The analysis of the processes described by system (1) follows these steps. 

Step 1. Modeling the process. 

Step 2. Determining the fixed points for (1). 

Step 3. Analyzing a fixed point of (1) by studying the sign of the characteristic equation of the 
linearized equation in the neighborhood of the fixed point. 

Step 4. Determining the value a. = o.q for which the characteristic equation has the roots 
]1\{dlq) = y(&o), f<2( a o) = F( a o) with their absolute value equal to 1, and the other roots with 
their absolute value less than 1 and the following formulas: 



da 



7^0, ^ Ko r^l,k = 1,2,3,4 

a=a a 



hold. 

Step 5. Determining the local center manifold Wf L c (0): 



— 1 2 -1-2 

zq + zq+ -iv 2 qz + w n zz + -w 02 z ,L + . 



where z = X\ + ix 2 , with (x.\,X2) G V\ C R , G V\, q an eigenvector corresponding to 
the eigenvalue y(0) and zi^O/^'ll'^CK are vectors that can be determined by the invariance 
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condition of the manifold Wf (0) with respect to the transformation x n - m = x ,..., x n = x'", 
x n+ i = x m+ . The restriction of system (1) to the manifold Wf (0) is: 

z„ +1 = ^(a )z„ + -g 2 o2« + gnz n z„ + -go22« + gnzlzn/2, (2) 

where g"20>Sll/g02/,§'21 are the coefficients obtained using the expansion in Taylor series 
including third-order terms of function /. 

System (2) is topologically equivalent with the prototype of the 2-dimensional discrete 
dynamic system that characterizes the systems with a Neimark-Sacker bifurcation. 

Step 6. Representing the orbits for system (1). The orbits of system (1) in the neighborhood of 
the fixed point x* are given by: 

x„ = x* + z„q + z„q + -r 2 o 2 « + fuz„z n + -r 02 zl (3) 

where z n is a solution of (2) and r^Q, T\\, Yq2 are determined with the help of u>20, U)\\, Wtyi- 
The properties of orbit (3) are established using the Lyapunov coefficient ?i(0). If /i(0) < 
then orbit (3) is a stable invariant closed curve (supercritical) and if /i(0) > then orbit (3) is 
an unstable invariant closed curve (subcritical). 
The perturbed stochastic system corresponding to (1) is given by: 

x n +i = j(Xn,x n — m ,tx) + g(x n ,x n — m )g n , (4) 

where x n = xq h , n € I = {—m, —m + 1, ..., —1,0} is the initial segment to be J^-measurable, 

and f„ is a random variable with E(f n ) = 0, E(£„) = c > and a is a real parameter. 

System (4) is called discrete-time stochastic system with delay. 

For the stochastic discrete-time system with delay, the stability in mean and the stability in 

square mean for the stationary state are done. 

This chapter is organized as follows. In Section 2 the discrete-time deterministic and 

stochastic dynamical systems are defined. In Section 3 the Neimark-Sacker bifurcation for 

the deterministic and stochastic Internet control congestion with discrete-time and delay 

is studied. Section 4 presents the deterministic and stochastic economic games with 

discrete-time and delay. In Section 5, the deterministic and stochastic Kaldor model with 

discrete-time is analyzed. Finally some conclusions and future prospects are provided. 

For the models from the above sections we establish the existence of the Neimark-Sacker 

bifurcation and the normal form. Then, the invariant curve is studied. We also associate 

the perturbed stochastic system and we analyze the stability in square mean of the solutions 

of the linearized system in the fixed point of the analyzed system. 

2. Discrete-time dynamical systems 

2.1 The definition of the discrete-time, deterministic and stochastic systems 

We intuitively describe the dynamical system concept. We suppose that a physical or biologic 
or economic system etc., can have different states represented by the elements of a set S. These 
states depend on the parameter t called time. If the system is in the state S\ G S, at the moment 
t\ and passes to the moment f 2 m the state s 2 £ S, then we denote this transformation by: 

*ti,f 2 ( s l) = s 2 
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and ^?ti,t 2 '■ S — > S is called evolution operator. In the deterministic evolutive processes the 
evolution operator ^ti,t 2 / satisfies the Chapman-Kolmogorov law: 

®h,t 2 ° \fi = ^W *f,f = "*S- 

For a fixed state Sg G S, application <t> : I? — > S, defined by t — > Sf = 4>t(srj), determines a 
curve in set S that represents the evolution of state So when time varies from — oo to oo. 
An evolutive system in the general form is given by a subset of S x S that is the graphic of the 
system: 

Fi{h>hi s ii s 2) = 0, i = 1..M 

where F, ■ : R 2 x S — > R. 

In what follows, the arithmetic space R"' is considered to be the states'space of a system, and 

the function <J> is a C -class differentiable application. 

An explicit differential dynamical system of C class, is the homomorphism of groups <t> : 

(R,+) -4 (Diff r (R m ),o) so that the application R x R m ->■ R m defined by (t,x) -> $(*)(*) 

is a differentiable of C r -class and for all x 6 7?'" fixed, the corresponding application 

4>(x) : R -»■ R m is C r+1 -class. 

A differentiable dynamical system on R m describes the evolution in continuous time of a 

process. Due to the fact that it is difficult to analyze the continuous evolution of the state 

Xq, the analysis is done at the regular periods of time, for example at t = — n, ..., — 1, 0, 1, ..., n. 

If we denote by <t>i = /, we have: 

*!(X ) =/(x ),*2(x ) =/ (2 '(*0) «M*0) =f {n \x Q ), 

O-itxo) =/ (_1) (xo) *-n(x ) =/ ( " B) (X ), 

where /< 2 ' = / of, ... ,/(») = /o ... o /,/"(») = /(- 1 ) o ... o/C" 1 ). 

Thus, <5 is determined by the diffeomorphism / = 3>j. 

A C-class differential dynamical system with discrete time on R'", is the homomorphism of 

groups $ : (Z,+) -4 (Diff(R m ),o). 

The orbit through Xq e R"' of a dynamical system with discrete-time is: 

O/^o) = {...,r (n) (x ) / ( - 1 )(xo) / x ,/(x ) / (n) (x ),..} = {/ (,!) (^o)}« e z. 

Thus 0<-(xq) represents a sequences of images of the studied process at regular periods of 
time. 

For the study of a dynamical system with discrete time, the structure of the orbits'set is 
analyzed. For a dynamical system with discrete time with the initial condition Xq G R m (m = 
1,2,3) we can represent graphically the points of the form x„ = /"(xg) for n iterations of the 
thousandth or millionth order. Thus, a visual geometrical image of the orbits'set structure 
is created, which suggests some properties regarding the particularities of the system. Then, 
these properties have to be approved or disproved by theoretical or practical arguments. 
An explicit dynamical system with discrete time has the form: 

x«+i =/(x n - p ,x„), n e N, (5) 

where / : R m x R"' — ► R'", x n G R'", p G N is fixed, and the initial conditions are x_ p , Xi_», 
..., x G R m . 
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For system (5), we use the change of variables x 
xP + = x n , and we associate the application 



K n-(p-l)' 



*n-l, 



rP+h 



G R'" x ... x R m -► R" 



xR" 



given by: 



( ^ \ 

xP 
\xP +1 / 



xP+ l 



Let (Q, J-) be a measurable space, where n is a set whose elements will be noted by cv and 

T is a a— algebra of subsets of fi. We denote by B( R) c— algebra of Borelian subsets of R. A 

random variable is a measurable function X : fi — > R with respect to the measurable spaces 

(fi, F) and (R,B{R)) (Kloeden et al., 1995). 

A probability measure P on the measurable space (fi, F) is a c— additive function defined on 

J 7 with values in [0, 1] so that P(fi) = 1. The triplet (£1, J ' , P) is called a probability space. 

An arbitrary family g(n, Co) = f (n)(to) of random variables, defined on fi with values in R, 

is called stochastic process. We denote £,(n,to) = f (n) for any n g N and a; G fi. The functions 

X(-, w) are called the trajectories of X(n). We use E(f(w)) for the mean value and E(£(n) ) 

the square mean value of f (m) denoted by ^„. 

The perturbed stochastic of system (5) is: 

x n+l =f(x n -p,Xn)+g(x„)£n, «GN 
where g : R n — > i?" and f n is a random variable which satisfies the conditions E(£ n ) = and 

E(e„) = <t>q. 



2.2 Elements used for the study of the discrete-time dynamical systems 

Consider the following discrete-time dynamical system defined on R" 1 : 



"-ji+1 



/(*») 



n e N 



(6) 



where / : _R m — > R 1 " is a C class function, called vector field. 

Some information, regarding the behavior of (6) in the neighborhood of the fixed point, is 

obtained studying the associated linear discrete-time dynamical system. 

Let i'o G R' n be a fixed point of (6). The system 



where 



«, !+ i = Df(x )u,„ n G N 



(^) 



is called the linear discrete-time dynamical system associated to (6) and the fixed point xq = /(xq). 
If the characteristic polynomial of D/(xq) does not have roots with their absolute values equal 
to 1, then i"o is called a hyperbolic fixed point. 
We have the following classification of the hyperbolic fixed points: 
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1. Xq is a stable point if all characteristic exponents of D/(xq) have their absolute values less 
than 1. 

2. Xq is an unstable point if all characteristic exponents of Df(xg) have their absolute values 
greater than 1. 

3. Xq is a saddle point if a part of the characteristic exponents of D/(xq) have their absolute 
values less than 1 and the others have their absolute values greater than 1. 

The orbit through Xq G R m of a discrete-time dynamical system generated by / : R'" — > R"' is 
stable if for any e > there exists S(e) so that for all x € B(xq,S(e)), d(f n (x) , f" {xq)) < e, for 
all n G N. 

The orbit through xq G R m is asymptotically stable if there exists 5 > so that for all x G 
B(i o ^),limrf(/»(i),/»(io)) = 0. 

If Xq is a fixed point of f, the orbit is formed by Xq. In this case O(xq) is stable (asymptotically 
stable) if d(f n (x),x ) < e, for all n G N and lim f n (x) = x . 

Let (CI, J-,P) be a probability space. The perturbed stochastic system of (6) is the following 
system: 

X n +\ = f{Xn) + g{Xn)£n 
where S, n is a random variable that satisfies E(f«) = 0, E(£„) = 0~ and g(xg) = with i"o the 
fixed point of the system (6). 

The linearized of the discrete stochastic dynamical system associated to (6) and the fixed point 
x is: 

M„ + l = Au„ + ZnBUn, ne N (7) 

where 

g)(*o), B=(f£|fo,), i,j = l..m. 

WeuseE(M n ) = E n , £(«««,!) = V„,u n = (u^u^, ...,u™) T . 

Proposition 2.1. (i) The mean values E„ satisfy the following system of equations: 

E, I+1 = AE„, neN (8) 

(ii) The square mean values satisfy: 

V„ +1 = AV n A T + aBV n B T , neN (9) 

Proof, (i) From (7) and E(£„) = we obtain (8). 
(ii) Using (7) we have: 

"h+i"k+i = Au n u T n A T + Z n (Au n iiJ,B T + Bu„u T n A T ) + l\Bu n u T n B T . (10) 

By (10) and E(f„) = 0, E(g) = <r we get (9). 

Let A be the matrix of system (8), respectively (9). The characteristic polynomial is given by: 

P 2 (A) =det(M- A). 

For system (8), respectively (9), the analysis of the solutions can be done by studying the roots 
of the equation -P2 W = 0. 
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2.3 Discrete-time dynamical systems with one parameter 

Consider a discrete-time dynamical system depending on a real parameter a, defined by the 
application: 

x->f(x,«.), xeR'",aeR (11) 

where / : R"' xR4 R'" is a seamless function with respect to x and a. Let xq 6 R m be a 
fixed point of (11), for all «6R. The characteristic equation associated to the Jacobian matrix 
of the application (11), evaluated in Xq is P(X, a) = 0, where: 

P(A, a) = A'" + q (a)A'"- 1 + • • • + c m _ 1 («)A + c m (a). 

The roots of the characteristic equation depend on the parameter a. 

The fixed point Xq is called stable for (11), if there exists a = «o so that equation P(A, «q) = 

has all roots with their absolute values less than 1. The existence conditions of the value o.q, 

are obtained using Schur Theorem (Lorenz, 1993). 

If m = 1, the necessary and sufficient conditions that all roots of the characteristic equation 

A + c\ (a)A + c 2 (a) = have their absolute values less than 1 are: 

|c 2 (a)|<l, |ci(a)| < |c 2 («) + l|. 
If m = 3, the necessary and sufficient conditions that all roots of the characteristic equation 

A 3 + c 1 0)A 2 + c 2 (a)A + c 3 (a) = 
have their absolute values less than 1 are: 

1 + ci(tx) + C2(«) + C3(«) > 0,1 — cj(a) + 02(0.) — cj,(&) > 
l+C2(a)-C3(a)(ci(a)+C3(a))>0 / l-c 2 (a)+C3(a)(c 1 (a)-c 3 (a))>0 / |c3(a)|<l. 

The Neimark-Sacker (or Hopf) bifurcation is the value a = «o for which the characteristic 
equation P(A,«rj) = has the roots jii(ocq) = }i(c(.q), }i2{ a o) — F(*o) i n their absolute values 
equal to 1, and the other roots have their absolute values less than 1 and: 



da 



i«=a 



7^0. b) }i k (a )=£l,k= 1,2,3,4 



hold. 

For the discrete-time dynamical system 

x(n + l) =f(x(n),«) 

with / : R"' — > R m , the following statement is true: 

Proposition 2.2. ((Kuznetsov, 1995), (Mircea et al., 2004)) Let ocq be a Neimark-Sacker bifurcation. 

The restriction of (11) to two dimensional center manifold in the point (xq, u.q) has the normal form: 

n^ t] e^(l + ^d\n\ 2 ) + 0(^ A ) 

where n S C, d € C If c =Re rf ^ there is a unique limit cycle in the neighborhood of xq. The 
expression of d is: 

d= - e - ie ° < v*,C(v,v,v)+2B(v,(I m - Ay l B(v,v)) + B(v,(e 2iea I m - A) { - l) B(v,v)) >0 



Discrete Deterministic and Stochastic Dynamical Systems with Delay - Applications 483 

where Av = e' e °v, A T v* = e~ t6a v* and < v*,v > = 1; A = ( -^- ) , B = ( — 4r ) and 

\ dx J(x oAo ) \dx 2 J(x oAa ) 

\ Sx3 J(x oAo ) 

The center manifold in xq is a two dimensional submanifold in R m , tangent in xq to the vectorial space 

of the eigenvectors v and v* . 

The following statements are true: 

Proposition 2.3. (i) If m = 2, the necessary and sufficient conditions that a Neimark-Sacker 
bifurcation exists in a. = ocq are: 

M«o)| = l, |ci(«o)|<2, ci(«o)^0, ci(«o)^l,^^| >0. 

(ii) If m = 3, the necessary and sufficient conditions that a Neimark-Sacker bifurcation exists in 
a = olq, are: 

|c 3 (a ) < 1, c 2 (a ) = l + Ci(a )c3(ao) -c 3 (a ) 2 , 
c 3 (a )(ci(ao)c3( a o) + ci(ao)c3(ao)-C2( a o) - 2c 3 {oc )c' 3 {a )) 



l + 2c 2 ,(a ) -ci(a )c3(«o) 
|c 1 (a )-c 3 (a )| t^O, |ci(«o) -c 3 («o)| 7^ 1- 



>a 



In what follows, we highlight the normal form for the Neimark-Sacker bifurcation. 

Theorem 2.1. (The Neimark-Sacker bifurcation). Consider the two dimensional discrete-time 
dynamical system given by: 

x->f(x,ot.), xeR 2 ,aeR (12) 

with x = 0, fixed point for all \a\ small enough and 

u 12 (cc) = r(«)e ±l >W 

where r(0) = 1, <p(0) = % If the following conditions hold: 

ci : r'(0)^0, c 2 : e iM » ^1, *:= 1,2,3,4 

ffrew f^ere /s a coordinates' transformation and a parameter change so that the application (12) is 
topologically equivalent in the neighborhood of the origin with the system: 



Vl \ /cos 6(6) -sin 0(6) 
y 2 J ^sin0(/3) cos 0(6) 



l S 



OCII + I 



+ (yf+y 2 )^ (/J) a ( 6)JU 2 ; 

where 0(0) = O , «(0) = Re(e -W °C 1 (0)) / flnd 

r rm = g*)(0)gu(0)(l-2?to) |gn(0)| 2 |gQ2(0)| 2 ai(0) 
U >~ 2(^- W ) " + 1-Fo 2(^_ Fo) + 2 

;<0 = e' e °, g20/ giv g02i gn are the coefficients obtained using the expansion in Taylor series including 
third-order terms of function f. 
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2.4 The Neimark-Sacker bifurcation for a class of discrete-time dynamical systems with 
delay 

A two dimensional discrete-time dynamical system with delay is defined by the equations 



Xn+1 = X„ +fl{x n ,y n ,a.) 
Vn+1 =y,i+f2(Xn-m,yn,lx) 



(13) 



where a € R, f\,f% '■ R —$■ R are seamless functions, so that for any |a| small enough, the 
system fi(x,y, ft) = 0, f2(x,y,a.) = 0, admits a solution (x,y) T 6 R 2 . 

Using the translation x n — > X n + x, y n — > y n +y, and denoting the new variables with the 
same notations x n , y„, system (13) becomes: 



X n +\ = X n +f{Xn,y n ,<X) 
Vn+1 =yn+g(x n -m,yn,<x) 



(14) 



where: 



f{x„,y„,a) = fi{x n + x,y„+y,a); g(x n - m ,y n ,0i) = f 2 {x n -m + x,y n + y,a) 

2 



With the change of variables x 1 = X n - m , x"- = X n _i m _\\, . . . , x 
y n , application (14) associated to the system is: 



v v m+l _ v v m+2 



( Xl \ 

v 2 



x m+l 

\x'»+ 2 J 



( 



\ 



:" ,+1 +f(x" ,+1 ,x m+2 ,a 
V x m+2 + g{x l ,x m+2 ,a) J 



(15) 



We use the notations: 



"10 
ho 



dx r ' 
*8 



,/ I (0 / / *) / fl i = ^ n y(0 / / «) 



" dx m + 2 



dx lK ' Ui dx m + 2 



(0,0, ft) 



"20 = 
fl02 = 
021 = 
%3 = 



dxm ?iL m+ i (°>°'«) 



d 2 f 



dx m+2 dx m + 2 
33/ 



(0,0, a) 



a\i 



«30 : 



9 2 / 



dx m+1 dx m + 2 

33/ 



dx m+l dx m+l dx m + 2 

3 3 / 

dx"'+ 2 dx"'+ 2 dx m + 2 



(0,0, a), a 12 -- 



(0,0, a) 



dx m+1 dx m+1 dx m+1 
dx m+l dx m+2 dx m + 2 



(0,0, ft), 

(0,0, ft), 
(0,0, a), 



(16) 
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fig 



'•'02 



dx m+2 dx m + 2 



(0,0, cc), 



21 dx l dx l dx m + 2 



6 

''30 

(0,0, a), b n 

(0,0, a). 



dx 1 dx m + 2 
dig 



dx l dx l dx l 



dx l dx m+2 dx m+2 



(0,0, a), 
(0,0, a), 

(0,0, a) 



(17) 



dx m + 2 dx m + 2 dx m + 2 
With (16) and (17) from (15) we have: 
Proposition 2.4. ((Mircea et al, 2004)) (i) The Jacobian matrix associated to (15) in (0, 0) T is: 



( 1 




\ 





. . . 1 + flio «oi 
V&irjO... l + fcoi/ 



(18) 



(ii) The characteristic equation of A is: 



i m+l 



A' n+Z - (2 + a w + b i)A'" +I + (1 + « 10 )(1 + fe 01 )A - fl 01 b 10 = 0. 



(19) 



(iii) If }i = u(a) is an eigenvalue of (19), then the eigenvector q G C m+2 , solution of the system 
Aq = uq, has the components: 



,i-i 



q 1 = l, q, = u , i = 2, . . . , m + 1, q m+2 



■'10 



V- - 1 - &oi 



(20) 



T/2£ eigenvector p g C'" + defined by A p = up has the components 



V\ 



(u.-l-a w ){}i-l-boi) 



^-r»i, i = 2, ...,m, 



(21) 



m(f - 1 - aio)(f - 1 - &oi) + K 2 F - 2 - «io - b 01 ) u 

i F 

u m L (}i-l-a w ) »io 

The vectors q, p satisfy the condition: 

m+2 
<(j,p>= £ q,p, - = 1. 
i=l 

The proof is obtained by straight calculation from (15) and (18). 

The following hypotheses are taken into account: 

Hj. The characteristic equation (19) has one pair of conjugate eigenvalues u(u), Jl(ci) with 
their absolute values equal to one, and the other eigenvalues have their absolute values less 
than one. 
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H2. The eigenvalues }i (a.), Ji(oi) intersect the unit circle for a = 0, and satisfy the transversality 
condition 

H 3 .Ifarg(^(a)) = 0(a), and O = 0(0), then e i9 « k ^ 1, fc = 1,2,3,4. 
From H2 we notice that for all |a| small enough, fi(tx) is given by: 

fi(ct) = r(oc)e ie ^ 

with r(0) = 1, 0(0) = O , r'(0) ^ 0. Thus r(a) = 1 + j8(«) where (6(0) = and j8'(0) 7^ 0. 
Taking /3 as a new parameter, we have: 

u(8) = (l + B)e ie W (22) 

with 0(0) = 0o- From (22) for /3 < small enough, the eigenvalues of the characteristic 
equation (19) have their absolute values less than one, and for B > small enough, the 
characteristic equation has an eigenvalue with its absolute value greater than one. Using 
the center manifold Theorem (Kuznetsov, 1995), application (15) has a family of invariant 
manifolds of two dimension depending on the parameter B. The restriction of application (15) 
to this manifold contains the essential properties of the dynamics for (13). The restriction of 
application (15) is obtained using the expansion in Taylor series until the third order of the 
right side of application (15). 

2.5 The center manifold, the normal form 

Consider the matrices: 

A = /«20«ll\ q _ ( «30 a l\\ D = /«21«12 
1 V fl H fl 02/' 1 V fl 21«12/' : \ rt 12 fl 03 



V hi hi) ' V h\ hi) ' V hi ^03 

with the coefficients given by (16) and (17). 

Denoting by x = (x 1 , . . .,x'" +2 ) e R m+2 , application (15), is written as x — > F(x), where 

F(x) = (x 2 , ..., x m , x m+1 + f{x m+1 , x m+2 , m) r x m + 2 + g{x x , x m + 2 , a) ). 

The following statements hold: 

Proposition 2.5. (i) The expansion in Taylor series until the third order of function F(x) is: 

F(x) = Ax+h(x,x) + l -C{x,x,x) + 0(|§| A ), (23) 

where A is the matrix (18), and 

B{x,x) = {0,...,0,B 1 {x,x),B 2 {x,x)) T , 
C{x,x,x) = (0,...,0,C 1 (x,x,x),C 2 {x,x,x)) T , 



(24) 
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where: 

B\x,x) = (x'"+\x m + 2 )A 1 (j^J ,B 2 (x,x) = (x\x'"+ 2 )A 2 (j? + . 

/ Y m+1 \ 

C l (x,x,x) = (x m+1 ,x m+2 )(x m+1 C 1 + x m+2 D 1 ) I m+2 J , 
C 2 (x,x,x) = (x\x m + 2 )(x l C 2 + x m + 2 D 2 ) (m+2) ■ 

(ii) Any vector x e R m+1 admits the decomposition: 

x = zq + zq + y, zgC (25) 

where zq + zq G T ce nter, y G T sta bi e ; T cen t e r is the vectorial space generated by the eigenvectors 
corresponding to the eigenvalues of the characteristic equation (19) with their absolute values equal to 
one and T staD i e is the vectorial subspace generated by the eigenvectors corresponding to the eigenvalues 
of the characteristic equation (19) with their absolute values less than 1. Moreover: 

z =< p,x >, y = x— < p,x > q— < p,x >q. (26) 

(iii) F(x) given by (23) has the decomposition: 

F(x) = F 1 (z,z) + F 2 (y) 
where 



Fi(z,z) = Gi(z)q + Gi(z 1 )q+ < p,N(zq + zq + y) > q+ < p,N(zq + zq + y) > q 

Gi(z) = pz+ < p,N(zq + zq + y) > (27) 

F 2 (y) = Ay + N(zq + zq + y)— < p,N(zq + zq + y) > q— < p,N(zq + zq + y) >q 

and 

N(zq + zq + y) = -B(zq + zq + y,zq + zq + y) + 

1 (28) 

+ -C(zq + z q + y, zq + z q + y, zq + z q + y) + 0(zq + zq + y) 

(iv) The two-dimensional differential submanifold from R' n+2 , given by x = zq + zq + V(z,z), z G 

- — - - dV(z,z) 

Vo C C, where V(z,z) = V(z,z), < p, V(z,z) >= 0, — ^ ' (0,0) = 0, is tangent to the vectorial 

Space T C enter in G C. 

Proof, (i) Taking into account the expression of F(x) we obtain the expansion in Taylor series 

until the third order (23). 

(ii) Because R" l+ = T cen t e r ffi T staD i e and < p,y >= 0, for any y G T s i a blei we obtain (25) and 

(26). 

(iii) Because F(x) G R m , with decomposition (25) and < p,q >= 1, < p,q >= 0, we have 

(27). 

(iv) Using the definition of the submanifold, this submanifold is tangent to T cen t er . 
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The center manifold in (0, 0) T 6 R 2 is a two dimensional submanifold from R m+1 tangent to 
^center at £ C and invariant with respect to the applications G\ and F2, given by (27). If 
x = zq + zq + V(z,z), 2 6 Vo C C is the analytical expression of the tangent submanifold to 
Tcenter/ the invariant condition is written as: 



V(G,(z),G 1 (z))=F 2 (V(z,z)). (29) 

From (27), (28) and (29) we find that x = zq + zq + V(z,z), z 6 Vq is the center manifold if 
and only if the relation: 

V(pz+ < p,N(zq + zq + V(z,z) >,pz+ < p,N(zq + zq + V(z,z)) >) = AV(z,z) + 
+ N(zq + zq + V(z,z))- < p,N(zq + zq + V(z,z)) > q- < p,N(zq + zq+V(z,z)) > q 

holds. 

In what follows we consider the function V (z, z) of the form: 

V{z,z) = -W2oz 2 + iv u zz + iv 02 z 2 + 0{\t\ 3 ), %eV, ed (31) 

Proposition 2.6. (i) If V(z, z) is given by (31), and N(zq + zq + y), with y = V(z, z) is given by 
(28), then: 

G\ (2) = F z + 2S20Z 2 + gnzz + g 02 z 2 + -g 2 iz 2 z + ... (32) 

where: 

g20=<P,B(q,q)>, g n =<p,B(q,-q)>, g 02 =< p,B{q,q) > 
gi\ =< p,B(q,w 2 o) > +2 < p,B(q,w n ) > + < p,C(q,q,q) > . 

(ii) IfV(z,z) is given by (31), relation (30) holds, if and only ifiV2o,Wn,WQ2 satisfy the relations: 

{p 2 I-A)w2o = h 2 o, {I-A)w n = h u , (p 2 I-A)w 02 = h 02 (34) 

where: 

h 20 = B(q,q)- < p,B(q,q) > q- < p,B(q,q) > q 
h n = B(q,q)— < p,B(q,q) > q— < p,B(q,q) > q 

h 02 = B(q,q)- < p,B(q,q) > q- < p,B(q,q) > q. 

Proof, (i) Because B(x,x) is abilinear form, C(x,x,x) is a trilinear form, and y = V(z,z), from 

(28) and the expression of G\ (z) given by (27), we obtain (32) and (33). 

(ii) In (30), replacing V(z,z) with (32) and N{zq + zq + V(z,z)) given by (28), we find that 

ro 20' w ll/ ^02 satisfy the relations (31). 

Let q G R m , p G R m be the eigenvectors of the matrices A and A T corresponding to the 

eigenvalues p and Ji given by (20) and (21) and: 

a = B 1 (q,q), b = B 2 (q,q), a\ = B 1 (q,q), b\ = B 2 (q,q),C\ = C 1 (q,q,q), C2 = C 2 (q,q,q), 
rl = B 1 {q,iv 20 ), rl = B 2 (^,w 20 ),r\ 1 = B 1 (q,w n ), r\ x = B 2 (q,w n ), 

(35) 
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where B , B , C , C , are applications given by (24). 

Proposition 2.7. (i) The coefficients g20- gu, g02 given by (33) have the expressions: 

g20 = Pm+ia + Vm+lb, gu = fm+l«l + Pm+lh, g02 = Pm+i* + Pm+lb- (36) 

(ii) The vectors h 2 Q,hn,hQ 2 given by (34) have the expressions: 

h 20 = (0,...,Q,a,b) T - (p m+1 a + p m+2 b)q- (p m+1 a + p m+2 b)q 

/in = (0,...,0,a,b) T - {p m+ iai + p m+2 bi)q - {p m+1 ai + p m+2 b)q (37) 

h 02 = h 20 . 

(iii) The systems of linear equations (34) have the solutions: 

„, _ ( 7 ,l „2„1 „2m„l a + {V 2 - aw)V lm v\o \ Pm+l" + Pm+lb Pm+l a + Pm+2 - 

W2 ° ~ [ V w * V ™ * V ™' ^ J — u q u 2 -Jl q 

( 1 1 1 «1 + (1 - «10)Pii ^ Pm+iai + Pm+2^1 „ Pm+l a l + Pm+2 b l - 

w v=[ v iv v n v iv ^ ) — t 1 T-f 1 

i _ flfloi -b(p 2 -b 01 ) J _ bifl i -fli(l-feoi) 



w 2 = w 20 ,v 20 = — 2 — , — 2 — r^^' p n 



{P' 2 -a w ){u 2 -b 01 )u 2m -b w a 01 ' (1 - a w )(l - b i) - b w a 01 ' 

(iv) Tfre coefficient g 2 \ given by (33) has the expression: 

g2\ = Pm+\r\o + Pm+ilw + HPm+lAl + Pm+2'll) + Pm+lA + P„, + 2C 2 . (38) 

Proof, (i) The expressions from (36) are obtained from (33) using (35). 

(ii) The expressions from (37) are obtained from (34) with the notations from (35). 

(iii) Because u ,~\r , 1 are not roots of the characteristic equation (19) then the linear systems 

(34) are determined compatible systems. The relations (37) are obtained by simple calculation. 

(iv) From (33) with (35) we obtain (38). 

Consider the discrete-time dynamical system with delay given by (13), for which the roots of 
the characteristic equation satisfy the hypotheses H\,H 2l Ho,. The following statements hold: 

Proposition 2.8. (i) The solution of the system (13) in the neighborhood of the fixed point (x,y) e R 2 , 
is: 

_ _ r m+\ 2 mW — m-\-l— 2 

X n = X -\- q m +\Z n -\- q m +\Zn + ^^20 ^ n ~*~ ^11 ZnZn ~^~ o"^02 ^« 

y n =y + q m+2 z„ + q m+2 z n + -^w^ 2 z\ + w™+ 2 z„z n + ^ioq 2 +2 zI (39) 

__ 12 1— 1 —2 

x n — m = U n = X -\- q\Z n + q^Z n + ;rW20 z « ~^~ w ll z n z n + ~ l "o2 2 n 



where z n is a solution of the equation: 



Z„+l = UZ n + ^OZn + gl\Z n Zn + ^g02zl + j_g2\z\z n (40) 
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and the coefficients from (40) are given by (36) and (38). 

(ii) There is a complex change variable, so that equation (40) becomes: 

w n+l = ^(/3)iy, 1 + C 1 (/3)^i(J H + 0(|3\| A ) (41) 

where: 

c (B) = g2 {f>)giiiP){Ti(P)-3-MP)) , |gu(fll 2 I^Q2(/3)| 2 g 2 i(/3) 

1[P > 2( F (/3) 2 -M/3))(K/3)-l) l-F(iS) + 2(^(/3)-)7(/3)) + 2 ' 

(iii) Let Iq = Re(e~ ie °Ci(0)), where 6q = arg(^(0)). Iflg < 0, in the neighborhood of the fixed point 
(x,y) there is an invariant stable limit cycle. 

Proof, (i) From Proposition 2.6, application (15) associated to (13) has the canonical form (40). 

A solution of system (40) leads to (39). 

(ii) In equation (40), making the following complex variable change 

2 = w + ^?0 w 2 Sn w - ^02 -2 + 

2(/^ 2 -f0 W-u 2(p2-p) 

£3° V+ g, 12 , WW 2 + J 03 ^ , 



6(1? -rr im 2 -}*) 6(f 3 -?o 

for fi small enough, equation (41) is obtained. The coefficients g2Q> glli gQ2 are given by (36) 
and 

£30 = Vm+\C l {q,q,q) + P,„+2C 2 {q,q,q), 
gl2 = Vm+\C l (q,q,q) + p m +2C 2 {q,q,q) 
g03 = Vm+\C l (q,q,q) + p m+2 C 2 {q,q,q). 

(iii) The coefficient Ci(/3) is called resonant cubic coefficient, and the sign of the coefficient Iq, 
establishes the existence of a stable or unstable invariant limit cycle (attractive or repulsive) 
(Kuznetsov, 1995). 

3. Neimark-Sacker bifurcation in a discrete time dynamic system for Internet 
congestion. 

The model of an Internet network with one link and single source, which can be formulated 
as: 

x(t) = k(w-af(x(t-T))) (42) 

where: k > 0, x(t) is the sending rate of the source at the time t, r is the sum of forward 
and return delays, w is a target (set-point), and the congestion indication function / : R+ — > 
R+ is increasing, nonnegative, which characterizes the congestion. Also, we admit that / is 
nonlinear and its third derivative exists and it is continuous. 
The model obtained by discretizing system (42) is given by: 

x„+i =x n - dkf(x n - m ) + kw (43) 

for n, m G N, m > and it represents the dynamical system with discrete-time for Internet 
congestion with one link and a single source. 
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Using the change of variables x = x n . 
associated to (43) is: 



Xfi~ 1/ x 



m+1 



x n , the application 



( ^ \ 



\ x m+l ) 



„m+l 



(44) 



\kw-akf(x 1 ) + x m+1 ) 



The fixed point of (44) is (x*, . . . , x*) T G R m+1 , where x* satisfies relation w = af(x*). With 
the translation x — > x + x* , application (44) can be written as: 



/ * \ 



\ x m+lJ 



r m+\ 



\kw-akg{x 1 )+x m+1 J 



where ^(x 1 ) = f(x l +x*). 
The following statements hold: 



Proposition 3.1. ((Mircea et al v 2004)) (i) The Jacobian matrix of '(45) in G R m+l is 



(45) 



( 
v- 


1 




. 

1 . 


. 



-akpi 


. 
. 


. 1 

■ 1/ 



where pi = g'(0). 

(ii) The characteristic equation of A is: 



A m+1 _ A m + akpi = 



(46) 



(47) 



(iii) If u G C is a root of (47), the eigenvector q G R m+l that corresponds to the eigenvalue u, of the 
matrix A, has the components: 

qi = u l ~ l , i = 1, . . . ,m + 1 

and the components of the eigenvector p G _R'" +1 corresponding to p. of the matrix A T are: 

akpi 1 ^ w 2 - u u 

n — ; — / Pi = -^-^rPlf I = 2,...,m — l,V m = ~Pi, Pm+l = r~ 

Ji m + 1 - makp x Fl p'- 1 ^ v pp x F v + akp- 



V\ 



Pi- 



?m+l 



m+1 



The vectors p G R , q G R satisfy the relation £ Vfli = !• 



;=i 



The following statements hold: 
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Proposition 3.2. (i)Ifm = 2, equation (47) becomes: 

A 3 - A 2 + akp x = 0. (48) 

Equation (48) has two complex roots with their absolute values equal to 1 and one root with the absolute 

a/5-1 y/§ - i 
value less than 1, if and only ifk = — . For k = kg = — , equation (48) has the roots: 

A u = exp (±»0(*d)9, 

9(«o) = arccos 1+ . (49) 

(ii) With respect to the change of parameter 

k = k(p)=k +g(fS) 

where: 

_ Vl + 4(1 + /3) 6 - (1 + /3) 2 - V5 + 1 
2/c 0jOl 

equation (49) becomes: 

A 3 - A 2 + afc(p>i = 0. (50) 

TTze roofs of equation (50) are: 

n M = (1 + /S)exp(±iw(/S)), A(/5) = -^fe 

where: 

,«, (l + /3) 2 + Vl + 4(l + /3)6 

"(0 = arccos i(TT^ ■ 

(iii) TTze eigenvector q 6 i? 3 , associated to the u = u(f>),for the matrix A has the components: 

<?1 = 1/ <?2 = F' </3 = F 2 

and f/ze eigenvector p e R 3 associated to the eigenvalue p = p(f>) for the matrix A T has the 
components: 

= ak Pl = ft 2 = g 

Pl 2akp x -Jl 3 ' F2 jp- 2akp x ' F3 jfi - 2ak Pl ' 
(iv) flg ' s a Neimark-Sacker bifurcation point. 
Using Proposition 3.2, we obtain: 
Proposition 3.3. The solution of equation (43) in the neighborhood of the fixed point x* e R is: 



U n = X* +Z„+Z„+ 2 W 20 z n + H>nZ«Z„ + 2 W 02 z n 

* — — 3 2 3— 3—2 

X n = X + q$Z n + (J3Z11 + ;rW>20 z « + w U z nZn + 7j w 02 z n 
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where: 



1 H \F 1J"20 \F -U"20 ' 20 2 21 il 3 41 2il i2 

-'20 = ^6 F 4 + flfcpi "'^20 = rtl - ft 20' ro 20 = J* W 20 " f ^20 " ^20 



ft?, 

7 11 ~ 



h\ Q = Aakp 2 {p 3 + p 3 ), h\ Q = akp l {p 3 q 2 + p i q 2 ),hl Q = 4akp 2 {l + p 3 q 3 + p 3 q 3 ) 

1,1 — 1,1 1,2 _ 1,2 1,3 _ l3 
"ll ~~ "20' "11 ~~ "20' "ll ~~ "20 

anrf z„6C is a solution of equation: 

z n+1 = uz n - -p 3 akp 2 (z 2 n + 2z„z„ + z 2 ) + -p^-kap^v^ - kapiio\ r + p 3 ), 

rho 1 =f'(p), p2=f"(P), p 3 =/'"(0). 
Let 

„ ,„-, »3fl 2 /c 2 pS(17 — 3 — 2u) fl 2 fc 2 p?|p 3 | 2 flfc|p 2 P3l 1 , i 

Cl( « = - V-^-1) + "W 1 - + 2(|^) + P3(-^-lo - -»Pl«4l + ft) 

and 

Z(0)=Ue(exp(-ifl(a ))C 1 (0)). 

Ifl{0) < 0, f/?e Neimark-Sacker bifurcation is supercritical (stable). 

The model of an Internet network with r links and a single source, can be analyzed in a similar 

way. 

The perturbed stochastic equation of (43) is: 

X n +1 =X„- Kkf(x n -m) + klO + £ n b{x n - X*) (51) 

and x* satisfies the relation w = af(x*), where E(£ n ) = 0, E(f 2 ) = a > 0. 

We study the case m = 2. Using (46) the linearized equation of (51) has the matrices: 

I ° 10 \ 
A-i= 1 ,B = 

\-akpi 1 / 

Using Proposition 2.1, the characteristic polynomial of the linearized system of (51) is given 
by: 

p 2 (A) = (A 3 - (1 + crb 2 )A 2 - a 2 k 2 pl){A 3 + akp x \ + a 2 k 2 p\). 

If the roots of P 2 (h) have their absolute values less than 1, then the square mean values of the 
solutions for the linearized system of (51) are asymptotically stable. The analysis of the roots 
for the equation ?2(A) = can be done for fixed values of the parameters. 
The numerical simulation can be done for: w = 0.1, a = 8 and f(x) = x 2 /(20 — 3x). 
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4. A discrete economic game with delay 

The economic game is described by a number of firms that enter the market with a 
homogeneous consumption product at different moments n, where n £ N. In what follows 
we consider two firms F\, F 2 and x, y the state variables of the model that represent the 
firms'outputs. The price function of the product (the inverse demand function) is p : R+ — > 
R+, derivable function with lim p(x) = 0, lim p(x) = 00 and p'(x) < 0. The cost functions 

X^co ,T->0+ 

are C, : R+ -> R+, i = 1,2, derivable functions with C\{x) ^ 0, C'/(x) > 0. The profit 
functions of the firms, 7T; : R\ — > R+, i = 1, 2, are given by: 

n x {x,y) = p(x + y)x - C t (x), n 2 {x,y) = p(x + y)y - C 2 (y). 

The non-cooperative game F\, F2, denoted by Y = (-R+, TC\, n^) is called deterministic economic 
game. The Nash solution of T is called the solution of the deterministic economic game. 
From the definition of the Nash solution, we find that the solution of the deterministic 
economic game is given by the quantities (x, y) € R\ for which the profit of each firm is 
maximum. Thus, the Nash solution is the solution of the following system: 



n lx = p 1 (x + y)x + p{x + y) - C[{x) = 

n 2y = p'{x + y)y + p{x + y) - c' 2 {y) = o. 

A solution (x, y ) T G R^_ of (52) is a (local) maximum for 7T„ i = 1, 2 if and only if: 
p"{x + y)x + 2p'{x + y) < C['(x), p"(x + y)y + 2p'(x + y) < Q"(y). 



(52) 



At each moment n, n 6 N the firms adjust their quantities x n ,y n , proportionally to the 
marginal profits -5 — , -= — . The quantities from the n + 1 moment satisfy the relations: 



(53) 



x n +i = x n +k(p'(x n +y n )x n + p(x n +y n )-C[{x n )) 

3/n+l = yn +l>c(p'(x„- m +y n )y n +P(x n -m + y n ) - C 2 (yn)) 

where m 6 N, m > 1. 

System (53) is a discrete dynamic economic game with delay. 

With respect to the change of variables x 1 = x„_ m , . . . , x m = x n _\, x m+1 = x n , x" l+2 = y n the 

application associated to (53) is: 



( xl \ 



x"' 

\ x nH2l 



( 



-nH-1 



x m + 1 +k(p'{x m + 1 +x m + 2 )x nH - 1 + p{x m + 1 +x nH2 )-C[{x nH - 1 )) 
V x" H2 +u{p , {x 1 +x ,m - 2 )x m + 2 + p{x 1 +x m + 2 )-C 2 {x m + 2 )) J 



(54) 



The fixed point of (54) is the point with the coordinates (xq, . . . , Xq, j/q) G R m where (xq, i/q) 
is the solution of the following system: 



p'(x + y)x + p(x + y) - C[(x) = 0, p'(x + y)y + p(x + y) - C 2 (y) = 



(55) 
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In what follows we use the notations: 

Pi = pW(x + y), y a = cj°(x), n% = C { 2 ] (y), i = 1,2,3,4 
the derivatives of i = 1, 2, 3, 4 order of the functions p, C\, Ci in the point (x, y), 



(56) 



«10 = Pl x + 2(0i - 7*21/ floi = P2 X + Pi, a 10 = P3* + 3pi - 7*31/ «11 = P3 X + 2 Pl, 

«02 = P3X + Pi, 030 = PiX + 4,03 - 7*41,821 = P\* + 3|0 3 , «12 = PiX + 2p 3 , fl 03 = p 4 X + 2p 3 , 



(57) 



&10 = PlV + Pi, % = PlV + 2(0i - 7*22, ho = P3V + Pi, hi = P3V + 2(02/ hi = P3V + 3(02 - 7*32/ 

ho = p4y+2p3,hi=p4y+2p3,hi = p4y+3p3,h3 = p4y+4:p3-n4i,di = b 01 +k{a w b 01 -a 01 b w ). 

(58) 

Proposition 4.1. ((Nearrvju, 2010)) (i) T/fg Jacobian matrix of the application (54) in (xq, . . . , xq, yg) 
is: 



( ° 



1 







\ 



... 
... 
\ab w 



1 

1 + Tcflio 7cfl i 
1 + ab i J 



(ii) The characteristic equation of A given by (59) is: 



ym+l _ aA m+l 



b\ m - c = 



where 



(59) 



a = 2 + ka w + ccb m , b = (l + ka w )(l + ocb m ), c = kKa 01 b w . 

(iii) The eigenvector a 6 R m+1 , which corresponds to the eigenvalue u of the matrix A and satisfies 
the system Aq = yq, has the components: 

ab w 



.J-1 



u ,1 = 1,..., m + 1, q 



m+l 



fi — 1 — ah 



01 



(60) 



The eigenvector p 6 R m+2 , which corresponds to the eigenvalue y of the matrix A and satisfies the 
system A T p = yp, has the components: 



Pi 



(u-l-ka w )(y-l-a.b w ) 



m{u - 1 - TcsioX?* - 1 - fcb 0i) + ?*(2?* - 2 - ka w - ab 01 ) 



Pi = =7rr, * = 2, . . . ,m - 2, p m+x = - m i,- — ~, — ; r 

W 1 y m 1 (ji-l-ka 10 ) 

The vectors q, p given by (60) and (61) satisfy the condition 

m+l 

<q,p>= E <?<?; = L 

1=1 



(61) 



Pi, Pm+l = -7—Pl- 

ab w 
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The proof follows by direct calculation. 
If m = 1, the following statements hold: 

b 
Proposition 4.2. (i) Ifk ^ — ^— and 

(k(a m b w - ferjl«ro) - b o\) 2 + 4fc«io«oi&io(&Ol - ka ik w ) > 
and olq is a solution of the equation: 

ka 01 b w (b 01 - ka m b w )a 2 + {k(a 01 b w - b 01 a w ) - b m )tx - ka w = 

so that: 

arjfc|«oifriol < 1/ |(fo i -fcfloi&io)«o + A + fcflio| < 2, 
then the equation: 

A 3 - rt A 2 + fc A - c = (62) 

/zas too roots with their absolute value equal to 1 and one root with its absolute value less than 1, where: 

«o = 2 + fcaio + «o&oi/ b o = {l + ka 10 )(l + oc b 01 ), c = ka 01 b w ix . (63) 

(ii) If for |/3 1 small enough, Aj(j8) > 0, where 

Ai(jS) = ((1 + £) 2 (fr ico + fcfloi&io«o) - 2c fcrt i&io - (1 + j5)%i(l + ^io)) 2 - 
-4fcfloifcio(fcoi(l+/3) 2 -fcfloi&io)((l+/5) 2 «oCo-c§ + (l+/3) 4 (l+rt c -c2) + (l+/3) 6 ), 

and aQ,bo,CQ given by (63), then there is g : R — > R, with g(0) = 0, g'{0) ^ so that the variable 
change: 

* = a(/3)=«o+g(jS) 
transforms equation (62) in equation: 

A 3 -a(p)A 2 + b(p)A-c{p) = (64) 

Equation (64) admits the solutions: 

* W (fl = (l + «*="<», e(/5 ) = arccos^|±^^,A (/5 ) = ^ (65) 

where: 



2c o fcfl O ibiQ+(l+/3) 4 boi(l+fcflio)-(l+/3) 2 (boic +fcflQifcoiflo)+VAi(^) 
^ W 2fc fl0 ifcio(fooi|l+/3| 2 -fcfloibio) 

fl(j8)=flO + feoi^(j8), &(0)=&O + Ml + *»lo)*O3), c(/3)=c +te ibio^(/5). 

For m = 1, the model of a discrete economic game with delay (53) is written as: 

x n +\ = x n +k(p'(x n +y n )x n + p(x n + y n ) - C[(x n )) 
J/n+1 = yn + «(p'(x„_i + y„)l/„ + p(x„_i + y„) - CjO/n))- 



(66) 
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We have: 

Proposition 4.3. (i) The solution of (66) in the neighborhood of (x, x, y) G R 3 is: 

X n = X + q 2 Z„ + q 2 Z„ + 2 W 20 z n + w \l z nZ n + lfv\ 2 z\, 

y n = y + q 3 Z„ + q 3 Z n + ^lo 2 ?! + w \l z n~Z n + w\ 2 z\, ( 67 ) 

u n = *„-i = x + q x z n + q^n + ^w\ Q z 2 n + w\-yZ n z n + w\ 2 z\, 

where z n is a solution of the equation: 

1 11 

Z„ = u(f>)z„ + ^giozl + gnZnZn + l^gwfn + ^gll^n (68) 

and the coefficients from (67) and (68) are given by (36) and (38) for m = 1, where 
fl i0/ fl 0i' fl 2(V fl ii' fl 02/ fl 30> fl 2i/ fl i2/ fl 03 are given by (57) and q 6 R 3 , p e R 3 are given by (60), (61) 
for m = 1. 

(ii) Iflo = Re(e~ ie (°>C\(0)), where 9(j3) is given by (65) and then ifl$<0 in the neighborhood of 

the fixed point (x,y) there is a stable limit cycle. 

For m = 2, the results are obtained in a similar way to Proposition 2.3. 

We will investigate an economic game where the price function is p(x) = -, i ^ 0, and the cost 
functions are Q(x) = CjX + b±, with c; > 0, i = 1,2. 

The following statements hold: 

Proposition 4.4. (i) The fixed point of the application (54) is (x, . . . , x, y) £ R m+2 where: 

c 2 



(ci + c 2 ) 2 ' 
(ci + c 2 ) 2 



y= ,„ ?„x 2 - ( 69 ) 



(ii) The coefficients (56), (57) and (58) are: 



p x = -(ci + c 2 ) 2 , p 2 = 2(ci + c 2 ) 3 , p 3 = -6(ci + c 2 f, Pi = 24(ci + c 2 f , 
«io = - 2c i( c i + c i), "oi = c \- c i«n = - 2 ( c i + c 2) 3 , ao2 = 2(ci + c 2 ) 2 (c 1 - 2c 2 ), 
a 20 = 6c 1 (c 1 + c 2 ) 2 , an = 12(ci + c 2 ) 3 (c 2 - c 1 ),a 2 i = 6(c x + c 2 ) 3 (c 2 - 3c\), a 21 = a u , 
a 30 = -24c 1 {c 1 + c 2 ) 3 ,b w = cl-4, b 01 = -2c 2 ( Cl +c 2 ), b 20 = 2{c x + c 2 ) 2 (c 2 - 2c x ), 
hi = 2(ci + c 2 ) 2 (2c 2 - c 1 ),b 02 = 6c 2 (c x + c 2 ) 2 , b 30 = 12(ci + c^ 3 ^ - c 2 ), b 21 = b 30 , 
bn = 6(ci + c 2 ) 3 ( Cl - 3c 2 ), b 03 = -24c 2 ( Cl + c 2 ) 3 . 

(iii) The solutions of system (66) in the neighborhood of the fixed point (x,x,y) G R 3 is given by (67). 
The coefficients from (67), (68) are: 

£20 = Pi a + Vi h > £n = P2«l + P3&1/ £02 = p 2 a + p 3 b, 
£21 = P2>20 + P3»20 + 2 (P2 r ll + P3>u) + VlC 1 + p 2 C 2 , 
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a = (<?2,<?3Mi(<?2,<?3) r , 
b = (qi,q 3 )A 2 (qi,q3) T , 

«i = {q2,q 3 )M{q 2 ,q 3 ) r , 

h = (qi,q3)A2(q~i,%) T , 
C 1 = (q 2 , <?3) ("72^11 + q 3 Au){q 2 ,q 3 ) J > cl = ("7l/?3)(<7l^21 + q 3 A 22 ){q 1 ,q 3 ) T , 

r 20 = fe^3) A l( a, 20' W 20) T ' r 20 = (?1> fa)^^, ro 2o) T ' 

r\ x = {q 2 ,q 3 )A 1 (wl v w 3 n ) T , r\ x = {q 1 ,q 3 )A 2 {zv\ 1 ,w 3 u ) T ' 

1 P2« + P3& P2 fl + P3 b 
n; 20 = p 2 j 7 =— / 

r - p r - v- 

2 2 p 2 a + p 3 b p 2 a + p 3 b_ 

W 20 = F^20 - P2 m2 "I qi - Vf-fe 

u u ¥■ V 

3 fl+(^ 2 -fli )^ 2 i>20 p 2 a+P3b . P 2 a+P 3 b 

W 2 ~ 9 ^3 7 ^^3/ 

sol f -f F -f 

ro 02 = ^20' ro 02 = ^20 w 02 = ^20 
1 P2«l + P3^1 p 2 g l + P3 fc l 

«>ii = "ii - -^-^ yz=-> 

w 2 = vu E^ + mh P2°l + P 3 h 

\ — ]l 1 — U ■ L 



Pi 



3 = fll + (l-fllo)^ll _ P2fll + P3frl _ P2 a l + P3 fc l - 

11 «oi i-F i-F 

(J7-l-fcflio)(77-l-afc 10 ) 

(Ji-1- ka w )(Jt - 1 - fc&oi) + f7(277 - 2 - ka w - nb 01 ) 



Pi f . «bio 

p 2 = _ 1 r- / P3 = tt-pv q\ = t qi = v, q?, - 



Ji-l-ka w ' ro ab w ri ' n ' " r ' '° u-l-ocb 01 ' 
(iv) The variations of the profits in the neighborhood of the fixed point (x, y) T € R 2 , are given by: 

Kin = p(x n + y,i)x,, - c x x n - b x , n 2n = p(x„ + y n )y n - c 2 y n - b 2 . 

The above model has a similar behavior as the economic models that describe the business cycles 

(Kuznetsov, 1995), (Mircea et al., 2004). 

The model can be analyzed in a similar way for the case m > 2. 

For m = 1, the stochastic system associated to (53) is given by: 

x„+i = x„ + k{p' (x n + y n )x n + p(x n + y„) - c[(x n )) + £ n b 22 (x n - x) 
y n +\ = Vn + fc(p'(x„_i + y n )y n + p{x n -\ + y n ) - c 2 {y,,)) + lnb 33 (y n - y) 
where (x,y) is the solution of (55). 
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The linearized of (70) has the matrices: 

/ 1 \ /0 \ 

A t = 1 + ka w ka m , B = b 22 (71) 

\ub w l + «*oi/ \° ° fo 33/ 

Using Proposition 2.1, the characteristic polynomial of (70) is given by: 

P 2 (A) = A(A 2 - A(fl 2 2fl33 + 0-b 22 b33) - A23fl33«3l)( A ( A - «22)( A ~ fl 33 - ^h) ~ fl 23 fl 3l)' ( 72 ) 

rofeere a 22 = 1 + fcflig, «23 = ^ fl oi' fl 3i = a ^i0' fl 33 = 1 + #&oi- 

T/ie analysis of the roots for the equation P 2 (\) = is done for fixed values of the parameters. 

The numerical simulation can be done for C\ = 0.1, c 2 = 0.4, k = 0.04, a = 0.4. 

5. The discrete deterministic and stochastic Kaldor model 

The discrete Kaldor model describes the business cycle for the state variables characterized by 
the income (national income) Y H and the capital stock K n , where n G JV. For the description 
of the model's equations we use the investment function I : R+ x i?+ — > R denoted by 
7 = I(Y,K) and the savings function S : R+ x R + -> R, denoted by S = S(Y,K) both 
considered as being differentiable functions (Dobrescu & Opri§, 2009), (Dobrescu & Opri§, 
2009). 

The discrete Kaldor model describes the income and capital stock variations using the 
functions I and S and it is described by: 

Y n +X = Y n + s(I(Yn,K n ) - S(Y n ,K n )) 
K n +x — K« + I{Y n ,K n ) — qK n . 

In (73), s > is an adjustment parameter, which measures the reaction of the system to the 
difference between investment and saving. 

We admit Keynes's hypothesis which states that the saving function is proportional to income, 
meaning that 

S(Y,K) = pY, (74) 

where p G (0, 1) is the propensity to save with the respect to the income. 

The investment function 7 is defined by taking into account a certain normal level of income 

pu 
u and a normal level of capital stock — , where u 6 R, u > 0. The coefficient q G (0,1) 

represents the capital depreciation. 

In what follows we admit Rodano's hypothesis and consider the form of the investment 

function as follows: 

I(Y,K) = pu + r(^-K\+f(Y-u) (75) 

where r > and / : R — > R is a differentiable function with /(0) = 0, /'(0) 7^ and 

/'"(0) ^ 0. 

System (73) with conditions (74) and (75) is written as: 
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Yn+i = (1 - sp)Y„ - rsK„ + sf{Y„ - u) + spu U + -) 
Kn+i = {l-r-q)K n + f(Y„-u) + pu U + -) 



(76) 



with s > 0, q 6 (0, 1), p e (0, 1), r > 0, !( > 0. 
The application associated to system (76) is: 



(1 — sv)v — rsk + sf(ii — u) + spu ( 1 H — 



(1 — sp)y — rsk + sf(y — u) + spu 1 + 
. (l-r-q)k + f(y-u) + pu(l+ r -j 



(77) 



The fixed points of the application (77) with respect to the model's parameters s, q, p, r are the 
solutions of the following system: 

py + rk - f(y - u) - pu ( 1 +'z) = ° 



(r + q)k- f(y - n) - pu(l + -\ 







that is equivalent to: 



qk-py = 0, p(l+ r -)(y-u)=f(y-u). 



(78) 



Taking into account that / satisfies /(0) = 0, by analyzing (78) we have: 



Proposition 5.1. (i) The point of the coordinates P I u, — ) is the fixed point of the application (77). 
(ii) If f(x) = arctanx, and p I 1 -\ — I > 1 then application (77) has an unique fixed point 

1 r. ( P u 

given by P I u, - — 

(iii) If f(x) = arctanx and p I 1 -\ — I < 1 then the application (77) has the fixed points 

P I u, — \, R iy r , ^- ), Q I y q , -y q \, where y q = 2u — y r and y r is the solution of the 



following equation: 



arctan(y — u) = p ( 1 + - J (y — u) 



Let (j/O/^o) be a fixed point of the application (77). We use the following notations: p\ 

f'(y - u), p 2 = /"(yo - u), p 3 = f"'(y - u) and 

fl io = s (i°i - V)- «oi = ~rs, b w = p lt b 01 = -q-r. 
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Proposition 5.2. (i) The Jacobian matrix of (77) in the fixed point (yo,ko) is: 

(ii) The characteristic equation of A given by (79) is: 

A 2 - aX + b = (80) 

where a = 2 + a w + 6 01/ 6 = 1 + a 10 + b 01 - floi^io- 

r(fl + v — 4) 
(iii) If a + r < 1, p\ < 1 + -^ — £ and s = so, where: 

fl + r 
so- 



(l-q-r)(p 1 -p)+r 

then equation (80) has the roots with their absolute values equal to 1. 
(iv) With respect to the change of variable: 

(l + /3) 2 -l + t? + r 
W (l-?-r)(ft-p)+r 

equation (80) becomes: 

A 2 -« 1 (/3)A + fo 1 (/3)=0 (81) 

where 

Equation (81) has the roots: 

fi,2(/5) = (l + /3) e ±,W 
where 

(v) The point s(0) = So is a Neimark-Sacker bifurcation point. 

(vi) The eigenvector q g R , which corresponds to the eigenvalue }i(fi) = ]i and is a solution 
of Ag = ]iq, has the components 

^ = 1, q 2= ^ l - a ^ . ( 82) 

"01 

The eigenvector p £ R , which corresponds to the eigenvalue Jl and is a solution of A p = Jlp, 
has the components: 

#01^10 

Pi - 



floi&io + (F- 1 - fl io) 2 ' 
fl 01 (^-l-a 01 ) 

"oi^io + (F- 1 - fl io) 2 



P2 - t^: — tt= — -■, — r^i- ( 83 ) 



The vectors q, p given by (82) and (83) satisfy the condition < q, p >= q\p l + qip 2 = 1- 
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The proof follows by direct calculation using (77). 

With respect to the translation y — > y + yo, k — > k + fcg, the application (77) becomes: 



(1 - sp)y - rsk + sf{y + y - u) - f{y - u) 
-(r + q)k + f(y + y -u)- f(y - u) 



(84) 



Expanding F from (84) in Taylor series around = (0,0) and neglecting the terms higher 
than the third order, we obtain: 



F(y,k) 



I 1 9 1 ^ \ 

I (I + a w )y + a 01 k + -sp 2 y + ^sp 3 y* \ 

1 , 1 o 

^ hoy + h\k + ^P2V + ^P3V J 



Proposition 5.3. (i) The canonical form of (84) is: 

z n+l = F(P)Zn + ^ ( s (£)Pl + P2)p2(zl + 2z„Z„ + Z„ 2 ) + 

+ g (s(j8)pi + P2)ps{4 + 3z 2 n z n + 3z„z2 + z\). 
(ii) The coefficient C\(f>) associated to the canonical form (85) is: 



(85) 



Ci(jS) 



(P(JS)PI + P2) 2 (F - 3 + 2j0 + |s(/3) P i + p 2 | 2 | 



2(^-u)(Ji-\) 

IK/3)Pl + P2| 2 \ , , s (/5)Pl+P2 



l-F 



2(f 2 -F) 



rf" 



f'3 



and /i(0) = Re(C 1 (0)e' fl (°)). If ^(0) < in the neighborhood of the fixed point (ij ,k ) then 
there is a stable limit cycle. If Zi (0) > there is an unstable limit cycle. 

(iii) The solution of (76) in the neighborhood of the fixed point (t/g, fcg) is: 

*n = 3/0 + Zii + z„, K„ = k + q 2 z» + q 2 z n 
where z„ is a solution of (85). 
The stochastic system of (76) is given by (Mircea et al., 2010): 



Yn+1 = (l-sp)Y n -rsK n + sf{Y„-u)+spu (l+-j +£nhl(Yn-u) 
(1-r- q)K n + f(Y„ - u) + pu (l + - ) + ^fc 22 (^ - — ) 

\ q j q 



K 



H + l 



with £(£„)= and E(^) = a. 

Using (79) and Proposition 5.2, the characteristic polynomial of the linearized system of (5) is 

given by: 



Discrete Deterministic and Stochastic Dynamical Systems with Delay - Applications 503 



P 2 (A) = det 



M- 


-(l + a w ) 2 -ab 2 n 


-«oi -2«oi(l + «io) 




-b\ Q A- 


-(l + b i) 2 -^22 -2fc 10 (l + fc 01 ) 




-fcio(l + aio) 


-«0l(l + &0l) A-(fl 01 b 10 + 

+ (l + fl 10 )(l + fe l) + 


V 




+ab n b 22 ) 



(86) 



The analysis of the roots for P 2 (A) = can be done for fixed values of the parameters. 

6. Conclusions 

The aim of this chapter is to briefly present some methods used for analyzing the models 

described by deterministic and stochastic discrete-time equations with delay. These methods 

are applied to models that describe: the Internet congestion control, economic games and the 

Kaldor economic model, as well. The obtained results are presented in a form which admits 

the numerical simulation. 

The present chapter contains a part of the authors' papers that have been published in journals 

or proceedings, to which we have added the stochastic aspects. 

The methods used in this chapter allow us to study other models described by systems of 

equations with discrete time and delay and their associated stochastic models. 
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1. Introduction 

The most apparent look of a discrete-time dynamical system is that an orbit is composed 
of a collection of points in phase space, in contrast to a trajectory curve for a 
continuous-time system. A basic and prominent theoretical difference between discrete-time 
and continuous-time dynamical systems is that chaos occurs in one-dimensional discrete-time 
dynamical systems, but not for one-dimensional deterministic continuous-time dynamical 
systems; the logistic map and logistic equation are the most well-known example illustrating 
this difference. On the one hand, fundamental theories for discrete-time systems have 
also been developed in a parallel manner as for continuous-time dynamical systems, 
such as stable manifold theorem, center manifold theorem and global attractor theory 
etc. On the other hand, analytical theory on chaotic dynamics has been developed more 
thoroughly for discrete-time systems (maps) than for continuous-time systems. Li-Yorke's 
period-three-implies-chaos and Sarkovskii's ordering on periodic orbits for one-dimensional 
maps are ones of the most celebrated theorems on chaotic dynamics. 

Regarding chaos theory for multidimensional maps, there are renowned Smale-Birkhoff 
homoclinic theorem and Moser theorem for diffeomorphisms. In addition, Marotto extended 
Li-Yorke's theorem from one-dimension to multi-dimension through introducing the notion 
of snapback repeller in 1978. This theory applies to maps which are not one-to-one (not 
diffeomorphism). But the existence of a repeller is a basic prerequisite for the theory. There 
have been extensive applications of this theorem to various applied problems. However, 
due to a technical flaw, Marotto fixed the definition of snapback repeller in 2005. While 
Marotto's theorem is valid under the new definition, its condition becomes more difficult 
to examine for practical applications. Accessible and computable criteria for applying this 
theorem hence remain to be developed. In Section 4, we shall introduce our recent works and 
related developments in the application of Marotto's theorem, which also provide an effective 
numerical computation method for justifying the condition of this theorem. 
Multidimensional systems may also exhibit simple dynamics; for example, every orbit 
converges to a fixed point, as time tends to infinity. Such a scenario is referred to as 
convergence of dynamics or complete stability. Typical mathematical tools for justifying such 
dynamics include Lyapunov method and LaSalle invariant principle, a discrete-time version. 
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However, it is not always possible to construct a Lyapunov function to apply this principle, 
especially for multidimensional nonlinear systems. We shall illustrate other technique that 
was recently formulated for certain systems in Section 3. 

As neural network models are presented in both continuous-time and discrete-time forms, 
and can exhibit both simple dynamics and complicated dynamics, we shall introduce some 
representative neural network models in Section 2. 

2. Neural network models 

In the past few decades, neural networks have received considerable attention and were 
successfully applied to many areas such as combinatorial optimization, signal processing 
and pattern recognition (Arik, 2000, Chua 1998). Discrete-time neural networks have been 
considered more important than their continuous-time counterparts in the implementations 
(Liu, 2008). The research interests in discrete- time neural networks include chaotic behaviors 
(Chen & Aihara, 1997; Chen & Shih, 2002), stability of fixed points (Forti & Tesi, 1995; Liang & 
Cao, 2004; Mak et al., 2007), and their applications (Chen & Aihara, 1999; Chen & Shih, 2008). 
We shall introduce some typical discrete-time neural networks in this section. 
Cellular neural network (CNN) is a large aggregation of analogue circuits. It was first 
proposed by Chua and Yang in 1988. The assembly consists of arrays of identical elementary 
processing units called cells. The cells are only connected to their nearest neighbors. This 
local connectivity makes CNNs very suitable for VLSI implementation. The equations for 
two-dimension layout of CNNs are given by 

dxjj l 

C-jf = - gXij(t) + 2^ [aijMKxkdt)) + h ijM u u] + 1 (i) 

(k,£)€Nij 

where u^, xu, h(xji) are the controlling input, state and output voltage of the specified 
CNN cell, respectively. CNNs are characterized by the bias I and the template set A and 
B which consist of a,-.- j^ and bu m, respectively, a,-.- ^ represents the linear feedback, and b;; ^ 
the linear control. The standard output h is a piecewise-linear function defined by /z(f) = 
i(|£ + l| — |£— 1|). Cis the linear capacitor and R is the linear resistor. For completeness 
of the model, boundary conditions need to be imposed for the cells on the boundary of the 
assembly, cf. (Shih, 2000). The discrete- time cellular neural network (DT-CNN) counterpart 
can be described by the following difference equation. 

*ij(t + 1) = mj(.t) + E fiij,k£ h (Xk£(t)) + hj,ki u ki\ + z i> ( 2 ) 

(M)gN, 7 

where t is an integer. System (2) can be derived from a delta operator based CNNs. If one 
collects from a continuous-time signal x(t) a discrete-time sequence x[k] = x(kT), the delta 
operator 

*x\k] = *t* + 1 j,-*M 

is an approximation of the derivative of x(t). Indeed, lim7-_j.o(5x[A:] = x(t)\ t= ^j. In this case, 
•fi = 1 — ^/ where T is the sampling period, and T = RC. The parameters ^i]Ui^i\U m (2) 
correspond to fl,,- jy, b,; jtf m (1) under sampling, cf. (Hanggi et al., 1999). If (2) is considered in 
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conjunction with (1), then T is required to satisfy T > T to avoid aliasing effects. Under this 
situation, < y, < 1. Thus CT-CNN is the limiting case of delta operator based CNNs with 
X — ¥ 0. If the delta operator based CNNs is considered by itself, then there is no restriction 
on T, and thus no restrictions on ]i in (2). On the other hand, a sampled-data based CNN 
has been introduced in (Harrer & Nossek, 1992). Such a network corresponds to the limiting 
case of delta operator based CNNs as T — > 1. For an account of unifying results on the 
above-mentioned models, see (Hanggi et al., 1999) and the references therein. In addition, 
Euler's difference scheme for (1) takes the form 

Xij(t + 1) = (1 - £c)*i/(*) + -q I E a ijM h( - x ke(t)) + bij M u u + l\. (3) 

Note that CNN of any dimension can be reformulated into a one-dimensional setting, cf . (Shih 

6 Weng, 2002). We rewrite (2) into a one-dimensional form as 

n 

Xi(t + 1) = jiXi(t) + £ cv ik h(x k {t)) + z,, (4) 

k=l 

The complete stability using LaSalle invariant principle has been studied in (Chen & Shih, 
2004a). We shall review this result in Section 3.1. 

Transiently chaotic neural network (TCNN) has been shown powerful in solving 
combinatorial optimization problems (Peterson & Soderberg, 1993; Chen & Aihara, 1995, 1997, 
1999). The system is represented by 

x t (t + 1) = iix,{t) + w u (t)[y,(t) - a oi ] + Zl_,iW ik y k {t) + a { (5) 

y l (t) = (l + e^)- 1 (6) 

w a (t + l) = (l- 7 )w u (t), (7) 

where i = 1, ■ ■ ■ , n, t £ IN (positive integers), e, 7 are fixed numbers with e > 0, < 

7 < 1. The main feature of TCNN contains chaotic dynamics temporarily generated for 
global searching and self-organizing. As certain variables (corresponding to temperature 
in the annealing process) decrease, the network gradually approaches a dynamical structure 
which is similar to classical neural networks. The system then settles at stationary states and 
provides a solution to the optimization problem. Equations (5)-(6) with constant self-feedback 
connection weights, that is, Wjj(t) = w^ = constant, has been studied in (Chen & Aihara, 
1995, 1997); therein, it was shown that snapback repellers exist if \wy\ are large enough. The 
result hence implicates certain chaotic dynamics for the system. More complete analytical 
arguments by applying Marotto's theorem through the formulation of upper and lower 
dynamics to conclude the chaotic dynamics have been performed in (Chen & Shih, 2002, 2008, 
2009). As the system evolves, zu„ decreases, and the chaotic behavior vanishes. In (Chen & 
Shih, 2004), they derived sufficient conditions under which evolutions for the system converge 
to fixed points of the system. Moreover, attracting sets and uniqueness of fixed point for the 
system were also addressed. 

Time delays are unavoidable in a neural network because of the finite signals switching and 
transmission speeds. The implementation of artificial neural networks incorporating delays 
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has been an important focus in neural systems studies (Buric & Todorovic, 2003; Campbell, 
2006; Roska & Chua, 1992; Wu, 2001). Time delays can cause oscillations or alter the stability 
of a stationary solution of a system. For certain discrete-time neural networks with delays, 
the stability of stationary solution has been intensively studied in (Chen et al., 2006; Wu et al., 
2009; Yua et al., 2010), and the convergence of dynamics has been analyzed in (Wang, 2008; 
Yuan, 2009). Among these studies, a typically investigated model is the one of Hopfield-type: 

m 

u,(t + 1) = Oi(t)Ui(t) + £ b i; -(t)«;(«;(* - '«/('))) + Ju i = l,2,---,m. (8) 

Notably, system (8) represents an autonomous system if fl/(f) = «;, and fo,;(f) = bu (Chen et 
al., 2006), otherwise, a non-autonomous system (Yuan, 2009). 

The class of Z-matrices consists of those matrices whose off-diagonal entries are less than 
or equal to zero. A M-matrix is a Z-matrix satisfying that all eigenvalues have positive real 
parts. For instance, one characterization of a nonsingular square matrix P to be a M-matrix is 
that P has non-positive off-diagonal entries, positive diagonal entries, and non-negative row 
sums. There exist several equivalent conditions for a Z-matrix P to be M-matrix, such as the 
one where there exists a positive diagonal matrix D such that PD is a diagonally dominant 
matrix, or all principal minors of P are positive (Plemmons, 1977). A common approach 
to conclude the stability of an equilibrium for a discrete-time neural network is through 
constructing Lyapunov-Krasovskii function/functional for the system. In (Chen, 2006), based 
on M-matrix theory, they constructed a Lyapunov function to derive the delay-independent 
and delay-dependent exponential stability results. 

Synchronization is a common and elementary phenomenon in many biological and physical 
systems. Although the real network architecture can be extremely complicated, rich 
dynamics arising from the interaction of simple network motifs are believed to provide 
similar sources of activities as in real-life systems. Coupled map networks introduced by 
Kaneko (Kaneko, 1984) have become one of the standard models in synchronization studies. 
Synchronization in diffusively coupled map networks without delays is well understood, 
and the synchronizability of the network depends on the underlying network topology 
and the dynamical behaviour of the individual units (Jost & Joy, 2001; Lu & Chen, 2004). 
The synchronization in discrete-time networks with non-diffusively and delayed coupling is 
investigated in a series of works of Bauer and coworkers (Bauer et al., 2009; Bauer et al., 2010). 

3. Simple dynamics 

Orbits of discrete-time dynamical system can jump around wildly. However, there are 
situations that the dynamics are organized in a simple manner; for example, every solution 
converges to a fixed point as time tends to infinity. Such a notion is referred to as 
convergence of dynamics or complete stability. Moreover, the simplest situation is that all orbits 
converge to a unique fixed point. We shall review some theories and results addressing 
such simple dynamics. In Subsection 3.1, we introduce LaSalle invariant principle and 
illustrate its application in discrete-time neural networks. In Subsection 3.2, we review the 
component-competing technique and its application in concluding global consensus for a 
discrete-time competing system. 
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3.1 Lyapunov method and LaSalle invariant principle 

Let us recall LaSalle invariant principle for difference equations. We consider the difference 
equation 

x(t + l) = F(x(f)), (9) 

where F : R" — > R" is a continuous function. Let U be a subset of R". For a function 
V : U — > R, define V(x) = V(F(x)) — V(x). V is said to be a Lyapunov function of (9) on 17 if 
(i) V is continuous, and (ii) V(x) < for all x G U. Set 

S := {xG U\V(x) = 0}. 

LaSalle Invariant Principle (LaSalle, 1976). Let F be a continuous mapping on R", and let V 
be a Lyapunov function for F on a set U C R". If orbit 7 := {F"(x)|ft G N} is contained in a 
compact set in U, then its o;-limit set a; (7) C So f] V (c) for some c = c(x). 

This principle has been applied to the discrete-time cellular neural network (4) in (Chen & 
Shih, 2004a), where the Lyapunov function is constructed as 

v ( x ) = ~\ E E co ik h( Xi )h(x k ) - £>/*(*-,) + Ul- F)t h (*,) 2 , 
L ;=ifc=i i=i z 1=1 

and ft(f) = |(|^ + 1| — |f — 1|) and x = (%\, ■ ■ ■ ,x n ) G R". Let us quote the main results 

therein. 

Proposition (Chen & Shih, 2004a). Let W be a positive-definite symmetric matrix and < }i < 

1. Then V is a Lyapunov function for (4) on R". 

Consider the condition 



(H) l 



1-u 



k,j k ="m" jk¥="m". k^=i 



> -1. 



Theorem (Chen & Shih, 2004a). Let W be a positive-definite symmetric matrix. If < u < 1 
and condition (H) holds, then the DT-CNN with regular parameters is completely stabile. 

Next, let us outline LaSalle invariant principle for non-autonomous difference equations. In 
addition to the classical result by LaSalle there is a modified version for the theorem reported 
in (Chen & Shih, 2004b). The alternative conditions derived therein is considered more 
applicable and has been applied to study the convergence of the TCNN. 
Let N be the set of positive integers. For a given continuous function F : N x R" — > R", we 
consider the non-autonomous difference equation 

x(t + l) = F(t,x(f)). (10) 

A sequence of points {x(f)}~inR" is a solution of (10) if x(f + 1) = F(f,x(f)), for all t G N. 
Let O x = {x(f) I t G N, x(l) = x}, be the orbit of x. We say that p is a w-limit point of 
O x if there exists a sequence of positive integers {t^} with tj. — > 00 as k — > 00, such that 
p = lirm-^oo x(fjt). Denote by tu(x) the set of all w-limit points of O x - 
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Let N,- represent the set of all positive integers larger than n,, for some positive integer n,-. Let 

G be any set in R" and G be its closure. For a function V : No x G — > R, define V(t,x) = 

V(t + l,F(f,x)) - V{t,x). If {x(t)} is a solution of (10), then V(t,x) = V(t + l,x(f + 1)) - 

V(t, x(f)). V is said to be a Lyapunov function for (10) if 

(i) { V(t, ■) | f € No} is equi-continuous, and 

(ii) for each p G G, there exists a neighborhood U oi p such that V(f, x) is bounded 

below for x£ LfnG and t G Ni, n\ > wq, and 

(iii) there exists a continuous function Qq : G — > R such that V(t, x) < — Qo( x ) < 

for all x G G and for all £ G N2, «2 > n l, 

or 

(iii)' there exist a continuous function Qq : G — )• R and an equi-continuous family of 

functions Q : N 2 x G -> R such that lim^oo | Q(t,x) - Q (x) | =0 for all x G G and 

V(£,x) < -Q(t,x) < for all (f,x) G N 2 x G,n 2 > n x . 

Define 

S = {xGG:Q (x)=0}. 

Theorem (Chen & Shih, 2004a). Let V : N x G ->■ R be a Lyapunov function for (10) and let 
O x be an orbit of (10) lying in G for all f G N . Then lim^co Q(f,x(f)) =0,ando;(x) C S . 

This theorem with conditions (i), (ii), and (iii) has been given in (LaSalle, 1976). We quote 
the proof for the second case reported in (Chen & Shih, 2004b). Let p G co(x). That is, 
there exists a sequence {f/c}i°, fjt — > °° as k — > 00 and x(fj-) — > p as k — > 00. Since 
y(fjt, x(f^)) is non-increasing and bounded below, Vftk, x((^)) approaches a real number as 
fc -». 00. Moreover, V(t k+V x(t k+1 )) - V(t v xfa)) < -E^" 1 Q(t,x(t)), by (iii)'. Thus, 
E&t 1 Q(*/X(f)) < 00. Hence, Q(f,x(f)) -> as t -+ 00, since Q(f,x(f)) > 0. Notably, 
Q(fj-,x(fj.)) — ^ Qo( x (^)) as fc — ^ °°- This can be justified by observing that 

IQ(*fc/x(*fc))-QoW*t))l 
< |Q(tfc,x(t fc )) + Q(*t/P) - Q(h,p) + Qo(p) - Qo(p) - Qo(x(tfc))l- 

In addition, |Q (x(f))| < |Q(t,x(t))| + \Q(t,x(t)) - Q (x(f))|. It follows from (iii)' that 

Qo( x {h)) — s- as A: — )• 00. Therefore, Qo(p) = 0' since Qo is continuous. Thus, p G So- 

If we further assume that V is bounded, then it is obvious that the proof can be much 

simplified. In the investigations for the asymptotic behaviors of TCNN, condition (iii)' is 

more achievable. 

We are interested in knowing whether if an orbit of the system (10) approaches an equilibrium 

state or fixed point as time tends to infinity. The structure of o;-limit sets for the orbits provides 

an important information toward this investigation. In discrete-time dynamical systems, the 

wTimit set of an orbit is not necessarily connected. However, the following proposition has 

been proved by Hale and Raugel in 1992. 

Proposition (Hale & Raugel, 1992). Let T be a continuous map on a Banach space X. Suppose 
that the w-limit set w(x) is contained in the set of fixed points of T, and the closure of the orbit 
O x is compact. Then w(x) is connected. 

This proposition can be extended to non-autonomous systems for which there exist limiting 
maps. Namely, 
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(A) There exists a continuous map F : R" —¥ R" such that lim^oo ||F(f, x) — F(x) || = 0, for all 
x G R". 

Theorem (Chen & Shih, 2004b). Assume that (10) satisfies (A), the orbit O x is bounded, 
and oj(x), the o;-limit set of x, is contained in the set of fixed points of F. Then co(x) is 
connected. Under this circumstances, if F has only finitely many fixed points, then the orbit 
O x approaches some single fixed point of F, as t tends to infinity. 

Let us represent the TCNN system (5)-(7) by the following time-dependent map 

F(f,x) = (F 1 (f,x),---,F„(f,x)) 
where 

n 

Fi(t,x) = ax, + (1 - 7) f w ;/ (0)(i/,- - floi) + tL^iflfj + 8 '' 

where y ( - = h i (x i ),i = 1, •• • ,n and h{ is defined in (6). The orbits of TCNN are then given 
by the iterations x(t + 1) = F(f,x(f)) with components x,(f + 1) = F,(f,x(f)). Note that 
y = H(x) = (h\ (xi), • • ■ ,h n (x n ) ) is a diffeomorphism on R" . Let Wg denote the n x n matrix 
obtained from the connection matrix W with its diagonal entries being replaced by zeros. 
Restated, W = W - diag[W] . For given < 7 < 1, choose < b < 1 such that | =^ | < 1. We 
consider the following time-dependent energy-like function: 

■inn n 

V{t,x) = --zYjH l W ii h ^ X i> h Mi> - *L a i h i( x i) 

+ (l-*)£ / %l hT^^drj + bK (11) 

Theorem (Chen & Shih, 2004b). Assume that Wq is a cycle-symmetric matrix, and either one 

of the following condition holds, 

(i) < a < I and Wq + 4(1 — a.)el is positive definite; 

(ii) j < a < 1 and Wq + 8ae7 is positive definite; 

(iii) a. > 1 and Wo + 8el is positive definite. 

Then there exists an Mg G N so that V(t, x) defined by (11) is a Lyapunov function for the 

TCNN (5)-(7) on N X R". 

3.2 Global consensus through a competing-component approach 

Grossberg (1978) considered a class of competitive systems of the form 

Xj = <7,-(x)[fc;(i-,-) -C(x 1 ,x 2 ,--- ,*„)], i = 1,2,- ■■ ,n, (12) 

where fl; > 0, dC/dXj > 0, x = [x\,X2, • • • ,x n ) G R". In such a system, n is the number of 
competing populations, fl,(x) refers to competitive balance, b;(x,-) represents interpopulation 
signal functions, and C(x) stands for mean competition function, or adaptation level. System 
(12) was proposed as a mathematical model for the resolution to a dilemma in science 
for hundred of years: How do arbitrarily many individuals, populations, or states, each 
obey unique and personal laws, succeed in harmoniously interacting with each other to 
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form some sort of stable society, or collective mode of behavior. Systems of the form (12) 
include the generalized Volterra-Lotka systems and an inhibitory network (Hirsch, 1989). A 
suitable Lyapunov function for system (12) is not known, hence the Lyapunov method and 
LaSalle invariant principle are invalid. The work in (Grossberg, 1978) employed a skillful 
competing-component analysis to prove that for system (12), any initial value x(0) > (i.e. 
x,(0) > 0, for any i) evolves to a limiting pattern x(oo) = (xi(oo),X2(oo), ■ ■ ■ ,x„(oo)) with 
< x,-(oo) := limt^oo Xj(t) < oo, under some conditions on a,-, bj, C. 

System (12) can be approximated, via Euler's difference scheme or delta-operator circuit 
implementation (Harrer & Nossek,1992), by 

Xi((k+1)5) = x,{k5) + dai(x(kd))[bi(xi(kS)) - C{x{kS))}, 

where one takes Xj(kS) as the fc-th iteration of x,. In this subsection, let us review the 
competing-component analysis for convergent dynamics reported in (Shih & Tseng, 2009). 
Consider the following discrete-time model, 

x,(k + l) = x,(k) +0a,-(x(fc))[b I -(x i (»O) - C(x(k))], (13) 

where i = 1,2, • • • , n, k e N := {0} (J N. We first consider the theory for (13) with f> = \, i.e. 

x i (fc + l) = z i (fc)+a i (x(fc))[b i (x i (fc))-C(x(fc))]. (14) 

The results can then be extended to (13). First, let us introduce the following definition for the 
convergent property of discrete-time systems. 

Definition. A discrete-time competitive system x(k + 1) = F(x(fc)) is said to achieve global 
consensus (or global pattern formation, global convergence) if, given any initial value x(0) G R", 
the limit x,(oo) := lim^oo Xj(k) exists, for all i = 1,2, • • • ,n. 

The following conditions are needed for the main results. 
Condition (Al): Each fl;(x) is continuous, and 

< fl,(x) < 1, for all x G R",f = 1,2,- ■■ ,n. 

Condition (A2): C(x) is bounded and continuously differentiable with bounded derivatives; 
namely, there exist constants M\, M2, T\ such that for all x £ R", 



Mi < C(x) < M 2 , 

C 
dx\ 



0< ^— (x) < r„ ;' = 1,2, ••• ,n. 
7<Xj > 



Condition (A3): b;(J) is continuously differentiable, strictly decreasing and there exist d, > 0, 
/;, Uj G R such that for all /' = 1, 2, ■ ■ ■ n, 

-di < &■(?) < 0, for all £e R, 

bi(S) > M 2 , for I < /„ and &,-(£ ) < Mi, for f > u,. 
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Condition (A4): For i = 1, 2, ■ ■ ■ , n, 

n 
< d t < 1 - £ rj < 1. 

7=1 

Theorem (Shih & Tseng, 2009). System (14) with ay by and C satisfying conditions (A1)-(A4) 
achieves global consensus. 

The proof of this theorem consists of three lemmas which depict the properties for the 
following terms: 

g,(k)=b i (x i (k))-C(x(k)), Ag,(k)=g,(k + l)-g,(k), 

g(k) = max{ gi (k) : i = 1,2,.. ■ ,n}, g(k) = mm{ gi (k) : i = 1,2,.. • ,«}, 

I(k) = min{i : fi (k) = |(fc)}, /(fc) = min{i : g,{k) = g{k)}, 

*W = X I(k)( k )' *W = X }(k)( k )' 

b(k) = b I{k) (x(k)),b(k) = b m (x(k)), 

Ab{k) = b{k + 1) - fe(Jc), Afo(Jc) = fo(Jc + 1) - b{k), 

Ab l (x l (k))=b l (x l (k + l))-b l (x,(k)). 

Let us recall some of the key lemmas to get a flavor of this approach. 

Lemma. Consider system (14) with «;, b\, and C satisfying conditions (A1)-(A4). Then 

(i) for function g, either case (g-(i)) or case (g-(ii)) holds, where 

(|-(i)): l(fc) < 0, for all k G N , 

(|-(ii)): g{k) > 0, for all k > K lt for some Kj G N ; 

(ii) for function ,g, either case (^-(i)) or case (£-(ii)) holds, where 

(|-(i)): l(fc) > 0, for all k G N , 

(^-(ii)): g(k) < 0, for all k > K 2 , for some K 2 G N . 

Lemma. Consider system (14) with ay &,-, and C satisfying conditions (A1)-(A4). Then 
lim^^eo^fc) = lim^eo C(x(fc)) = lim^^oo^fc). 

4. Complicated dynamics 

In this section, we summarize some analytic theories on chaotic dynamics for 
multi-dimensional maps. There are several definitions for chaos. Let us introduce the 
representative one by Devaney (1989): 

Definition. Let (X, d) be a metric space. A map F : O C X — > O is said to be chaotic on fi if 

(i) F is topologically transitive in fi, 

(ii) the periodic points of F in fi are dense in fi, 

(iii) F has sensitive dependence on initial conditions in fi. 

It was shown in (Banks, et al., 1992) that condition (iii) holds under conditions (i) and (ii), if F 
is continuous in fi. Let us recall Li-Yorke's theorem. 
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Theorem (Li & Yorke, 1975). Let / be an interval and let / : / — > J be continuous. Assume 
there is a point a £ / for which the points b = f(a), c = f (a) and d = f (a), satisfy 

d < a <b < c { or d > a > b > c). 

Then for every k = 1, 2, ■ ■ ■ , there is a periodic point in / having period k. Furthermore, there 
is an uncountable set S C / (containing no periodic points), which satisfies: 
(i) limsup,,^ |/»(p) - f"(q)\ > 0, and liminf„^oo \f"(p) - f"(cj)\ = 0, for every p,q £ S 
with p 7^ q; (ii) limsup H ^ co |/"(p) — /"(<?) I > 0/ for every p e S and periodic point q e J. 
Indeed, if there is a periodic point of period 3, then the hypothesis of the theorem will be 
satisfied. The notion of scrambled set can be generalized to metric space (X,d). 

Definition. Let (X, d) be a metric space and F : X — > X be a continuous map. A subset S of X 
is called a scrambled set of F, if for any two different points x, y £ S, 

liminfd(F"(x),F"(y)) = 0, limsupd(F"(x),F"(y)) > 0. 

A map F is said to be chaotic in the sense of Li-Yorke if it has an uncountable scrambled set. It 
was shown in (Huang & Ye, 2002) that for a compact matric space (X, d), if a map F is chaotic 
in the sense of Devaney then F is also chaotic in the sense of Li-Yorke. 
Let us consider a differentiable map 

F : R" ->• R"; (15) 

we denote x^ = F* : (xg) for k e N and xq £ R", and by B r (x) and B*(x) the closed balls in 
R" with center at x and radius r > under Euclidean norm || ■ || and certain norm || • ||*, 
respectively. 

Definition. Suppose z is a hyperbolic fixed point of a diffeomorphism map F : R" — > R™, and 
some eigenvalues of DF(z) are greater than one in magnitude and the others smaller than one 
in magnitude. If the stable manifold and the unstable manifold of F at z intersect transversally 
at some point xq, the orbit {xt}/^ = _ co of F is called a transversal homoclinic orbit. 
For a diffeomorphism F, Smale discovered an elegant and significant result: 

Theorem (Smale, 1967). If the diffeomorphism map F has a transversal homoclinic orbit, then 
there exists a Cantor set A C R" on which F m is topologically conjugate to a full shift of a 
symbolic dynamical system with N symbols, for some positive integer m. 

Remark. The above theorem can be generalized to maps which are not diffeomorphisms 
under some extended definition of transversal homoclinic orbits, see Theorem 5.2 and Section 
7 in (Hale & Lin, 1986) and Theorem 5.1 in (Steinlein & Walther, 1990). 

4.1 On Marotto's theorem 

Analytical theory on chaotic dynamics for multi-dimensional systems is quite limited; yet 
some important progresses have been made. In 1978, Marotto introduced the notion of 
snapback repeller and extended Li-Yorke's theorem to multi-dimensional maps. This result 
plays an important role in the study of chaos for higher but finite-dimensional noninvertible 
maps. 
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The point z G R" is called an expanding fixed point of F in B r (z), if F is differentiable in B,-(z), 
F(z) = z and 

|A(x)| > 1, for all eigenvalues A(x) of DF(x), for all x g B,(z). (16) 

If F is not a one-to-one function in R" and z is an expanding fixed point of F in B r (z), then 
there may exist a point xq £ B r (z) with xq 7^ z such that F (xrj) = z for some positive integer 
£. The original definition of snapback repeller is as follows. 

Definition (Marotto, 1978). Assume that z is an expanding fixed point of F in B r (z) for some 
r > 0. Then z is said to be a snapback repeller of F if there exists a point xq £ B r (z) with xq 7^ z, 
F (xrj) = z and det(DF (xrj)) 7^ for some positive integer £; see Fig. 1. 

F(x ) = X! 

F(x 1 ) = x 2 




^(•^/-2/ — "*7-l 

Fig.l 

It is straightforward to see that a snapback repeller gives rise to an orbit { x k}T=-co °^ ^ wn "h 
xj- = z, for k > £, and x^ — ¥ z as fc — > —00. Roughly speaking, the property of this orbit 
is analogous to the one for homoclinic orbit. In addition, the map F is locally one-to-one at 
each point xj-, since xq 6 B,-(z) and det(DF (xrj)) 7^ 0. This leads to the trivial transversality, 
i.e., the unstable manifold R" of full dimension intersects transversally the zero- dimensional 
stable manifold of z. Therefore, snapback repeller may be regarded as a special case of a fixed 
point with a transversal homoclinic orbit if the latter is generalized to mappings which are not 
one-to-one. 

Theorem (Marotto, 1978). If F possesses a snapback repeller, then F is chaotic in the following 

sense: There exist (i) a positive integer N, such that F has a point of period p, for each integer 

p > N, (ii) a scrambled set of F, i.e., an uncountable set S containing no periodic points of F, 

such that 

(a) F(S) C S, 

(b)limsup fc ^ 00 ||F Jc (x)-F ,c (y)|| > 0, forallx,y g S,withx^y, 

(c) limsupj.^^ || F (x) — F (y)|| > 0, for all x g S and periodic point y of F, 

(iii) an uncountable subset So of S, such that liminfj-^oo \\F (x) — F (y)|| = 0, for every x, y £ 

So- 
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Remark. As the implication of this theorem yields the existence of uncountable scrambled set, 

we may say that existence of snapback repeller implies chaos in the sense of Li-Yorke. 

However, there is a technical flaw in the original derivation. Consider the following two 

statements: 

(A): All eigenvalues of the Jacobian DF(z) are greater than one in norm. 

(B): There exist some s > 1 and r > such that 

||F(x)-F(y)|| >s||x-y||, forallx,y£ B,-(z). (17) 

That (A) implies (B) may not be true for the Euclidean norm in multi-dimension. In addition, 
if z is a fixed point and there exists a norm || ■ || *, such that 

|A(x)| > 1, for all eigenvalues A (x) of DF(x), for all x G B*(z), 

then 

||F(x)-F(y)||* >s-||x-y||*, for all x, ye B*{z), (18) 

still may not be satisfied. This is due to that the norm constructed for such a property depends 
on the matrix DF(x) which varies at different points x, as the mean-value inequality is applied. 
Several researchers have made efforts in modifying the definition of snapback repeller to 
validate the theorem. In 2005, Marotto gave a revised definition of snapback repeller. Note 
that a fixed point z of F is repelling if all eigenvalues of DF(z) exceed one in norm. For a 
repelling fixed point z, if there exist a norm || - || * on R" and s > 1 such that (18) holds, then 
B* (z) is called a repelling neighborhood of z. Note that if z is a repelling fixed point of F, then one 
can find a norm || • || * and r > so that B* (z) is a repelling neighborhood of z, see (Robinson, 
1999). 

Definition (Marotto, 2005). Let z be a repelling fixed point of F. Suppose that there exist a 
point xq 7^ z in a repelling neighborhood of z and an integer £ > 1, such that X( = z and 
det(DF(xjt)) 7^ for 1 < k < (,. Then z is called a snapback repeller of F. 

The point xg in the definition is called a snapback point of F. While Marotto's theorem holds 
under the modified definition, its application becomes more inaccessible; indeed, it is a 
nontrivial task to confirm that some preimage of a repelling fixed point lies in the repelling 
neighborhood of this fixed point. From practical view point, condition (16) which was 
adopted in his original version, is obviously easier to examine than finding the repelling 
neighborhood for a fixed point. In (Liao & Shih, 2011), two directions have been proposed 
to confirm that a repelling fixed point is a snapback repeller for multi-dimensional maps. 
The first one is to find the repelling neighborhood U of the repeller z which is based on a 
computable norm. This is the key part in applying Marotto's theorem for practical application, 
as one can then attempt to find a snapback point xq of z in U, i.e., F (xrj) = z, xg 6 U and 
Xq ^= z, for some i > 1. The second direction is applying a sequential graphic-iteration 
scheme to construct the preimages {z_j.}?Li of z, such that F(z_^) = z_j. +1 , k > 2, F(z_j) = 
z, \in\f c _ tco F(z_f c ) = z. Such an orbit {z_j-}?°, is a homoclinic orbit for the repeller z, in 
the generalized sense, as mentioned above. The existence of such a homoclinic orbit leads 
to the existence of a snapback point in the repelling neighborhood of repeller z. Therefore, 
without finding the repelling region of the fixed point, Marotto's theorem still holds by using 
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the second method. More precisely, two methodologies were derived to establish the existence 

of snapback repellers: 

(i) estimate the radius of repelling neighborhood for a repelling fixed point, under Euclidean 

norm, 

(ii) construct the homoclinic orbit for a repelling fixed point by using a sequential 

graphic-iteration scheme. 

In some practical applications, one can combine (i) and (ii) to achieve the application 

of Marotto's theorem. These two methodologies can then be combined with numerical 

computations and the technique of interval computing which provides rigorous computation 

precision, to conclude chaotic dynamics for the systems, such as the transiently chaotic neural 

network (TCNN) and the predator-prey system (Liao & Shih, 2011). Let us recall the results 

therein. 

Repelling neighborhood: 

Proposition (Liao & Shih, 2011). Consider a continuously differentiable map F : R" — > R" 

with fixed point z. Let 



Si := \l minimal eigenvalue of (DF(z)) T DF(z) , 



?/,-:= max ||B(w,z)||2 
weB,.(z) 



= max \ maximal eigenvalue of (B(w,z)) r B(w,z), 
weB r (z) v 

where B(w, z) := DF(w) — DF(z). If there exists a r > such that 

si - rj r > 1, (19) 

then B,-(z) is a repelling neighborhood for z, under the Euclidean norm. 

There is a second approach which is based on the estimate of the first and second derivatives 
of F. This estimate is advantageous for quadratic maps since their second derivatives are 
constants. Let C;(x) and B,,(x) be defined as 



u,(x) := v/ eigenvalues of (DF(x)) r DF(x), 
6,;(x) := eigenvalues of Hessian matrix Hy. (x) = [3jfc9/Fj( x )]itxZ/ 

where i, j = 1, 2, ■ ■ ■ , n. Let tx r and S r be defined as 

Di r := min min jcfx)} (20) 

xeB r (z)l<K« 

& r := max max max 1 6,-,-fx) I. (21) 

l<i<nxeB r (z)l<j<n ' 

Proposition (Liao & Shih, 2011). Consider a C 2 map F = (F lr ■ ■ ,F n ) : R" -)■ R" with fixed 
point z. Let ct r and fi r be defined in (20) and (21). If there exists r > 0, such that 

a r - rVnjSr > 1, (22) 
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then B r (z) is a repelling neighborhood of z, under the Euclidean norm. 

The conditions (19) and (22) are computable numerically and the value r can be found from 
numerical computation. Furthermore, if there exists a snapback point xq in B,-(z), i.e., xq G 
B,-(z), and F (xq) = z for some integer £ > 1, then z is a snapback repeller. Hence, the map F 
is chaotic in the sense of Marotto. 

Sequential graphic-iteration scheme: 

We recall an approach which is developed to exploit the existence of snapback 
repeller, without estimating the repelling neighborhood. In particular, it is a scheme 
to construct homoclinic orbits for repelling fixed point x of F: {x, : j G N} with 
F(x_l) = x, F(x_;) = x ; + i, for j > 2, and lim.-^^, F(x ;) = x. 

Theorem (Liao & Shih, 2011). Assume that there exists a compact, connected, convex region 
n = nf =1 fi; C R", so that the C 1 map F = (Fi,F 2/ • • • ,F„) : R" -)■ R" satisfies 

?)F- " r)F- 

I^WI>!+ E l^(x)|, forall/=l,...,n, xen, 

and has a repelling fixed point xinflc R". For i = 1, ■ ■ ■ , «, set 

/, /(1) (f) := sup{F,(ii- ■ ■ ,x\_ v ?„x' i+v -- ■ ,x' n ) : x) G n y ,; G {1,- ■ ■ ,«}/{/}}, 
/ i(1) (f) := inf{F,(ii- ■ ■ ,^_ 1/ f,x{ +1/ . • • ,4) : ^ e n /V ; G {1,- • • ,n}/{»}}, 

for ^ G R 1 . Also assume that /; m\ and /; n-, both have fixed points in d,-, for all i = 1, ■ ■ ■ , n, 
and 

x_ f+ l G R" \ n, x_ e G int(n) 

hold, for some £ > 2. Then there exist a sequence of nested regions (Om j^Lj with fl/^i C 
f!m C n, and preimages x_)t_i G flm of x under F, k G IN. If furthermore, ||Q, m|| — > 0, as 
fc — > oo, for all i = 1, • • • , n, then {x_^}^_ 1 is a homoclinic orbit for x. Moreover, if 

det(DF(x_jt)) t^O, fori <k< £-1 

holds, then x is a snapback repeller and F is chaotic in the sense of Marotto's theorem. 

Remark, (i) The existence of this homoclinic orbit guarantees the existence of the snapback 
point without finding the repelling neighborhood, (ii) The conditions in the above theorem 
are formulated for DF and the one-dimensional maps /, m(£) an d // m (£) (the upper and 
lower maps), hence they are easy to examine in applications. For example, for TCNN map, 
we can find explicit and computable conditions such that all conditions in the theorem are 
satisfied. 

4.2 Applications and extensions 

We review some applications of snapback repeller and chaotic dynamics in (Marotto, 1979a, 
1979b). Consider a two-dimensional mapping F : R 2 — > R 2 of the form F(x,y) = (f(x),x), 
with / : R — > R being differentiable. 
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Lemma (Marotto, 1979a). (i) If / has a stable periodic point z of period p, then F(x,y) = 
(f(x),x) has a stable periodic point (z, yrj) or period p where yo = fV (z). (ii) If / has a 
snapback repeller, then F(x, y) = (f(x), x) has a transversal homoclinic orbit. 
Using these results, one can investigate the dynamics of the following difference equation: 

x k+i = f(xk,bxk-i), (23) 

where b,x k g R and / : R — > R is differentiable. We rewrite (23) into the following 
two-dimensional system: 

x k+i = f(xk, tyk) 

(24) 

3/Jc+l = x k- 
Moreover, when b = 0, (23) is the following scalar problem 

**+l =/(**, 0). (25) 

It was shown that the dynamics of (23) or (24) are determined by those of (25), if b is close to 
0: 

Theorem (Marotto, 1979a). (i) If (25) has a stable periodic point Xq of period p, then there 
exists e > such that (24) has a stable periodic point (x(b), y(b)) of period p for all \b\ < e. In 
this case (x(b),y(b)) is a uniquely defined, continuous function of b with x(0) = Xq. (ii) If (25) 
has a snapback repeller, then (24) has a transversal homoclinic orbit for all \b\ < e, for some 
e > 0. 

Next, let us consider another class of two-dimensional map G : R — \ R which is determined 
by two scalar equations fix) and g(y) where f,g : R — ¥ R are differentiable and G(x, y) is 
defined by G{x, y) = {f{x),g{y)). 

Lemma (Marotto, 1979a). (i) If one of the mappings / and g has a snapback repeller and the 
other has an unstable fixed point, then G(x,y) = (f(x),g(y)) has a snapback repeller. (ii) If 
one of the mappings / and g has a snapback repeller and the other has a stable fixed point, 
then G(x,y) = (f{x),g(y)) has a transversal homoclinic orbit. 

Now, we consider the dynamics for systems of the form: 

x k+i = f{x k ,by k ) 

(26) 
Vk+l = g{cx k ,y k ), 

where /, g : R — > R are differentiable, and b, c £ R are close to 0. If b = c = 0, then (26) can 
be simplified to the uncoupled system: 

x*+l =/(**, 0) (27) 

yjt+i = *(o,y*). (28) 

Theorem (Marotto,1979a). (i) If one of the (27) and (28) has a snapback repeller and the other 
has an unstable fixed point, then (26) has a snapback repeller for all | b\, \ c\ < e,forsomee > 0. 
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(ii) If one of the (27) and (28) has a snapback repeller and the other has a stable fixed point, 
then (26) has a transversal homoclinic orbit for all \b\, \c\ < e, for some e > 0. 

Remark. By examining the simplified systems, the above results exhibit the dynamics of 
system (24) or (26) under some small perturbations of certain parameters. However, these 
theorems do not provide any indication about the estimate of e. 

Next, let us recall the Henon map 

*k+i = Vk + 1 - ax l 
Vk+1 = bx k , 
which can be equivalently written as 

M/c+1 = bv k + l- au\ =: f(u k ,bv k ) 



Vk+l = u k , 



(29) 



where f{u,v) = v + 1 — au 2 . It was shown in (Marotto, 1979b) that u k+ i = f(u k ,0) has a 
snapback repeller, when a > 1.55. Hence (29) has a transversal homolinic orbit for all a > 1.55 
and \b\ < e, for some e > 0. 

In (Li, et al., 2008), they considered a one-parameter family of maps H^ on R" x W" with 
H (x,y) = (F(x),G(x)) and continuous F : R" ->• R" and G : R" ->■ R'" or H (x,y) = 
(F(x),G(x,y)) with continuous maps F : R" ->• R" and G : R" x R m ->■ R". They used 
the covering relations method proposed by Zgliczyhski in (Zgliczyhski, 1996, 1997) to prove 
that if n = 1 and F has a positive topological entropy, or if n > 1 and F has a snapback 
repeller, then any small perturbation H\ of Hq has a positive topological entropy. Without 
using hyperbolicity, the covering relations method still provides a way to verify the existence 
of periodic points, the symbolic dynamics and the positive topological entropy. Moreover, 
they also applied this method to obtain a new proof for Garcia's result (Garcia, 1986) that if a 
map has a snapback repeller then it has a positive topological entropy. One can obtain similar 
results by using this method with other structure, such as a hyperbolic horseshoe. 
Since the definition of snapback repeller proposed by Marotto relies on the norm, the 
following definition independent of norm was proposed. 

Definition (Li, et al., 2008). Let F : R" -)■ R" be a C 1 function. A fixed point z for F is called a 
snapback repeller if (i) all eigenvalues of the derivative DF(z) are greater than one in absolute 
value and (ii) there exists a sequence {x_,}, £ ]n such that x_j ^ z, lim^oo x_,- = z, and for all 

i e N, F(x_j) = x_ /+1 , F(x_ x ) = z and det(DF(x_,-)) ^ 0. 

Remark. Although the above definition is independent of norm on the phase space, it requires 
the existence of the pre-images for the repeller. The sequential graphic-iteration scheme 
outlined above provides a methodology for such a construction. 

Note that item (i) implies that there exist a norm || ■ ||* on R", r > and s > 1, such that 
||F(x) — F(y)||* > s||x — y||* for all x,y <= B*(z). Hence F is one-to-one on B*(z) and 
F(By(z)) D B*(z). Therefore, if there exists a point xg £ BJ^(z) such that F (xq) = z and 
det( DF(xq)) 7^ for some positive integer (.., then item (ii) of the above definition is satisfied. 
In addition, in (Li & Chen, 2003), they showed that this norm can be chosen to be the Euclidean 
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norm on R", under the condition that all eigenvalues of (DF(z) ) DF(z) are greater than one. 
However, this condition is more restrictive, due to that a repelling fixed point has the potential 
to be a snapback repeller, without satisfying this condition. 

Theorem (Li, et al., 2008). Let H\ be a one-parameter family of continuous maps on R" x R'" 
such that H^(x, y) is continuous as a function of A G R and (x, y) G R" x R'". Assume 
that Hq(x, y) = (F(x),G(x)), where F : R" —¥ R" is C 1 and has a snapback repeller and 
G : R" — > R m . Then H\ has a positive topological entropy for all A sufficiently close to 0. 

Theorem (Li, et al.,2008). Let H\ be a one-parameter family of continuous maps on R" x R"' 
such that Ff^(z) is continuous as a function of A G R and (x, y) 6 R" x R'". Assume that 
H (x,y) = (F(x), G(x,y)), where F : R" -> R" is C 1 and has a snapback repeller, G : R" x 
R"' -> R'", and G(R" x S) C int(S) for some compact set S C R"' homeomorphic to the 
closed unit ball in R'". Then H^ has a positive topological entropy for all A sufficiently close 
toO. 

Moreover, it was shown in (Li & Lyu,2009) that if F has a snapback repeller and G is a small C 1 
perturbation of F, then G has a snapback repeller, positive topological entropy, as the implicit 
function theorem is applied. Moreover, G is chaotic in the sense of Li-Yorke. More precisely, 

Theorem (Li & Lyu, 2009). Let F be a C map on R" with a snapback repeller. If G is a C 
map on R" such that \\F — G\\ + \\DF — DG||* is small enough, where || ■ ||* is the operator 
norm on the space of linear maps on R" induced by the Euclidean norm || ■ ||, then G has a 
snapback repeller, exhibits Li-Yorke chaos, and has positive topological entropy. 

Corollary (Li & Lyu, 2009). Let F^,(x) be a one-parameter family of C maps with x G R" 
and u G R . Assume that F^(x) is C as a function jointly of x and }i and that Ff, has a 
snapback repeller. Then map F„ has a snapback repeller, exhibits Li-Yorke chaos, and has 
positive topological entropy, for all u sufficiently close to Uq- 

In (Shi & Chen, 2004, 2008), they generalized the definitions of expanding fixed point, 
snapback repeller, homoclinic orbit, and heteroclinic orbit for a continuously differentiable 
map from R" to general metric spaces as follows. Herein, Bj? (x) denotes the closed balls of 
radius r centered at x G X under metric d, i.e. 

Bf.{z) := {xGR" :d(x,z) < r}. 

In the following, we introduce the coupled-expanding map. 

Definition (Shi & Chen, 2008). Let F:Dc X->Xbea map where (X, d) is a metric space. If 
there exists £ > 2 subsets V„ 1 < i < £, of D with V { n Vi = d D V t n d D Vj for each pair of (i,/), 
l<i^j<£, such that 

F(Vi) D UJ =1 Vj, l<i<£, 

where do^i is the relative boundary of V, with respect to D, then F is said to be 
coupled-expanding in Vj, 1 < i < £. Moreover, the map F is said to be strictly coupled-expanding 
in V„ l<i<e,ii d(V„ Vj) > 0, for all 1 < i ^ j < £. 

Definition (Shi & Chen, 2004). Let F : X — > X be a map on metric space (X, d). (i) A point 
z G X is called an expanding fixed point (or a repeller) of F in Bf (z) for some constant rrj > 0, if 
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F(z) = z and 

d{F{x),F{y)) > Ad(x,y), for all x, y G B*(z) 

for some constant A > 1. Moreover, z is called a regular expanding fixed point of F in B{? (z) if z 
is an interior point of F(int(BJ? (z))). 

(ii) Let z be an expanding fixed point of F in BJ? (z) for some rg > 0. Then z is said to be a 
snapback repeller of F if there exists a point xq G int(Bj? (z)) with xj ^ z and F (xrj) = z for 
some positive integer £ > 2. Moreover, z is said to be a nondegenerate snapback repeller of F if 
there exist positive constants u and Jg such that int( Br (xq)) C int(BJ? (z)) and 

d(F*(x),F*(y)) > ud(x,y), for all x,y G B^xg); 

z is called a regular snapback repeller of F if F(int(BJ? o (z))) is open and there exists a positive 
constant^ such that int(B|,( x o)) C int(BJ?(z)) and z is an interior point of F^(int(Bjj(xg))) 
for any positive constant 5 < 5%. 

(iii) Assume that z 6 X is a regular expanding fixed point of F. Let U be the maximal open 
neighborhood of z in the sense that for any x G U with x ^= z, there exists /eg > 1 with 
F ka (x) £ U, F~ k (x) is uniquely defined in 17 for all k > 1, and F~ k (x) — > z as k — > oo. !I is 
called the local unstable set of F at z and is denoted by YJf (z). 

(iv) Let z G X be a regular expanding fixed point of F. A point x G X is called homoclinic to z if 
x G W," (z), x^z, and there exists an integer m > 1 such that F'"(x) = z. A homoclinic orbit 
to z, consisting of a homoclinic point x with F'"(x) = z, its backward orbit {F^x)}^, and 
its finite forward orbit {F-' (x) j™^ , is called nondegenerate if for each point x; on the homoclinic 
orbit there exist positive constants r, and ft; such that 

d(F(x),F(y)) > Ujd(x,y), forallx,y G B?.{xj). 

A homoclinic orbit is called regular if for each point x, on the orbit, there exists a positive 
constant f, such that for any positive constant r < J;, F(x.) is an interior point of 
F(int(BJ?(x,))). Otherwise, it is called singular. A point x is called heteroclinic to z, if x G Wj" c (z) 
and there exists a m > 1 such that F'" (x) lies on a different periodic orbit from z. 
Notice that if a map F on R" has a snapback repeller, and is continuously differentiable in 
some neighborhood of x; = F^(xg), for < j < I — 1, then the snapback repeller is regular 
and nondegenerate. For continuously differentiable finite-dimensional maps, the definition 
of snapback repeller has been extended in (Shi & Chen, 2004, 2008) to the maps in the general 
metric space, through introducing the two classifications: regular and singular, nondegenerate 
and degenerate. It was proved that a map F is a strict coupled-expansion and chaotic in the 
sense of both Devaney and Li-Yorke if F has a nondegenerate and regular snapback repeller 
or a nondegenerate and regular homoclinic orbit to an expanding fixed point. Moreover, if F 
is C 1 in R" and has a snapback repeller under Marotto's definition, then the snapback repeller 
is nondegenerate and regular. Therefore, F is chaotic in the sense of Marotto, Devaney, and 
Li-Yorke. In addition, more general scenario for degenerate and regular snapback repeller, 
was studied in (Shi & Yu, 2008). 
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4.3 Some remarks 

We summarize some results concerning the above-mentioned notions. 

(i) For a compact metric space (X, d), chaos in the sense of Devaney implies chaos in the sense 

of Li-Yorke. 

(ii) If a map F has a snapback repeller z, then F is chaotic in the sense of Marotto and Li-Yorke. 

(iii) If a map F : A — > A is topologically conjugate to the symbolic dynamical system o~ : E J — > 

Lj" / ror some Cantor set A, then F is chaotic on A in the sense of Devaney and Li-Yorke. 

(iv) For a complete metric space (X,d) and a map F : X — > X, if F has a regular nondegenerate 

snapback repeller z £ X, then there exists a Cantor set A so that F'" : A — ► A is topologically 

conjugate to the symbolic dynamical system o~ : E^~ — > E~" / ror some integer m. Consequently 

F m is chaotic on A in the sense of Devaney and Li-Yorke. 

(v) For a complete metric space (X, d) and a C map F : X — > X, if F has a Marotto's snapback 

repeller z, then z is also a regular nondegenerate snapback repeller. Hence, F m is chaotic in 

the sense of Devaney and Li-Yorke, for some integer m. 

(vi) If a map F has a transversal homoclinic orbit, then there exists a Cantor set A so that 

F m : A —)• A is topologically conjugate to the symbolic dynamical system o~ : E^ — > Ej, for 

some integer m. Consequently, F"' is chaotic on A in the sense of Devaney and Li-Yorke. 
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